1832 lines
No EOL
53 KiB
Markdown
1832 lines
No EOL
53 KiB
Markdown
# Datamodels and Interfaces Component
|
|
|
|
## Overview
|
|
|
|
The Datamodels and Interfaces components form the core data layer of the Gateway application. They provide a clean separation between data structures (datamodels) and data access logic (interfaces), enabling type-safe, maintainable, and scalable data operations throughout the application.
|
|
|
|
## Component Architecture
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Application Layer"
|
|
App[Application<br/>Routes, Services, Features]
|
|
end
|
|
|
|
subgraph "Interfaces Layer"
|
|
IF_RealEstate[Real Estate Interface]
|
|
IF_Chat[Chat Interface]
|
|
IF_App[App Interface]
|
|
IF_Component[Component Interface]
|
|
IF_AI[AI Interface]
|
|
IF_Ticket[Ticket Interface]
|
|
IF_Voice[Voice Interface]
|
|
end
|
|
|
|
subgraph "Access Control Layer"
|
|
AC_RealEstate[Real Estate Access]
|
|
AC_Chat[Chat Access]
|
|
AC_App[App Access]
|
|
AC_Component[Component Access]
|
|
end
|
|
|
|
subgraph "Database Connectors"
|
|
Connector_Postgre[PostgreSQL Connector]
|
|
end
|
|
|
|
subgraph "Databases"
|
|
DB_RealEstate[(Real Estate<br/>Database)]
|
|
DB_Chat[(Chat<br/>Database)]
|
|
DB_App[(App<br/>Database)]
|
|
DB_Component[(Component<br/>Database)]
|
|
end
|
|
|
|
subgraph "External Systems"
|
|
External_AI[AI APIs<br/>OpenAI, Anthropic, etc.]
|
|
External_Tickets[Ticket Systems<br/>Jira, ClickUp]
|
|
External_Voice[Voice Services<br/>Google Cloud]
|
|
end
|
|
|
|
App -.->|uses| IF_RealEstate
|
|
App -.->|uses| IF_Chat
|
|
App -.->|uses| IF_App
|
|
App -.->|uses| IF_Component
|
|
App -.->|uses| IF_AI
|
|
App -.->|uses| IF_Ticket
|
|
App -.->|uses| IF_Voice
|
|
|
|
IF_RealEstate --> AC_RealEstate
|
|
IF_Chat --> AC_Chat
|
|
IF_App --> AC_App
|
|
IF_Component --> AC_Component
|
|
|
|
AC_RealEstate --> Connector_Postgre
|
|
AC_Chat --> Connector_Postgre
|
|
AC_App --> Connector_Postgre
|
|
AC_Component --> Connector_Postgre
|
|
|
|
Connector_Postgre --> DB_RealEstate
|
|
Connector_Postgre --> DB_Chat
|
|
Connector_Postgre --> DB_App
|
|
Connector_Postgre --> DB_Component
|
|
|
|
IF_AI --> External_AI
|
|
IF_Ticket --> External_Tickets
|
|
IF_Voice --> External_Voice
|
|
|
|
IF_RealEstate -.->|uses| DM_RealEstate[Real Estate<br/>Datamodels]
|
|
IF_Chat -.->|uses| DM_Chat[Chat<br/>Datamodels]
|
|
IF_App -.->|uses| DM_UAM[User & Mandate<br/>Datamodels]
|
|
IF_Component -.->|uses| DM_Files[File<br/>Datamodels]
|
|
IF_AI -.->|uses| DM_AI[AI<br/>Datamodels]
|
|
```
|
|
|
|
## Data Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Route as API Route
|
|
participant Service as Service Layer
|
|
participant Interface as Interface
|
|
participant Access as Access Control
|
|
participant Connector as Database Connector
|
|
participant DB as Database
|
|
|
|
Route->>Service: Request with User Context
|
|
Service->>Interface: Initialize with User
|
|
Interface->>Access: Check Permissions
|
|
Access-->>Interface: Permission Granted
|
|
Service->>Interface: CRUD Operation
|
|
Interface->>Access: Validate Access
|
|
Access-->>Interface: Access Validated
|
|
Interface->>Connector: Execute Query
|
|
Connector->>DB: SQL Query
|
|
DB-->>Connector: Result Set
|
|
Connector-->>Interface: Data Objects
|
|
Interface->>Access: Apply Filtering
|
|
Access-->>Interface: Filtered Data
|
|
Interface-->>Service: Datamodel Instances
|
|
Service-->>Route: Response Data
|
|
```
|
|
|
|
## Component Structure
|
|
|
|
### Datamodels Structure
|
|
|
|
```
|
|
modules/datamodels/
|
|
├── datamodelRealEstate.py # Real estate domain models
|
|
├── datamodelChat.py # Chat workflow models
|
|
├── datamodelAi.py # AI operation models
|
|
├── datamodelUam.py # User and mandate models
|
|
├── datamodelSecurity.py # Security and authentication models
|
|
├── datamodelFiles.py # File management models
|
|
├── datamodelDocument.py # Document structure models
|
|
├── datamodelExtraction.py # Content extraction models
|
|
├── datamodelPagination.py # Pagination models
|
|
├── datamodelVoice.py # Voice settings models
|
|
├── datamodelTickets.py # Ticket system models
|
|
├── datamodelNeutralizer.py # Data neutralization models
|
|
├── datamodelTools.py # Tool definitions
|
|
├── datamodelUtils.py # Utility models
|
|
└── __init__.py # Package exports
|
|
```
|
|
|
|
### Interfaces Structure
|
|
|
|
```
|
|
modules/interfaces/
|
|
├── interfaceDbRealEstateObjects.py # Real estate data access
|
|
├── interfaceDbRealEstateAccess.py # Real estate access control
|
|
├── interfaceDbChatObjects.py # Chat data access
|
|
├── interfaceDbChatAccess.py # Chat access control
|
|
├── interfaceDbAppObjects.py # App/user management access
|
|
├── interfaceDbAppAccess.py # App access control
|
|
├── interfaceDbComponentObjects.py # Component management access
|
|
├── interfaceDbComponentAccess.py # Component access control
|
|
├── interfaceAiObjects.py # AI operations interface
|
|
├── interfaceTicketObjects.py # Ticket system interface
|
|
└── interfaceVoiceObjects.py # Voice operations interface
|
|
```
|
|
|
|
---
|
|
|
|
## Datamodels Component
|
|
|
|
### datamodelRealEstate.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
Projekt {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string statusProzess
|
|
json perimeter
|
|
json baulinie
|
|
json parzellen
|
|
json dokumente
|
|
json kontextInformationen
|
|
}
|
|
|
|
Parzelle {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string kontextGemeinde FK
|
|
json perimeter
|
|
json baulinie
|
|
string bauzone
|
|
float az
|
|
float bz
|
|
json dokumente
|
|
json kontextInformationen
|
|
}
|
|
|
|
Land {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string abk
|
|
json dokumente
|
|
json kontextInformationen
|
|
}
|
|
|
|
Kanton {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string id_land FK
|
|
string abk
|
|
json dokumente
|
|
json kontextInformationen
|
|
}
|
|
|
|
Gemeinde {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string id_kanton FK
|
|
string plz
|
|
json dokumente
|
|
json kontextInformationen
|
|
}
|
|
|
|
Dokument {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string versionsbezeichnung
|
|
string dokumentTyp
|
|
string dokumentReferenz
|
|
string quelle
|
|
string mimeType
|
|
json kategorienTags
|
|
}
|
|
|
|
Kontext {
|
|
string id PK
|
|
string thema
|
|
string inhalt
|
|
}
|
|
|
|
GeoPunkt {
|
|
string koordinatensystem
|
|
float x
|
|
float y
|
|
float z
|
|
string referenz
|
|
}
|
|
|
|
GeoPolylinie {
|
|
string id PK
|
|
bool closed
|
|
json punkte
|
|
}
|
|
|
|
Projekt ||--o{ Parzelle : contains
|
|
Projekt ||--o{ Dokument : references
|
|
Projekt ||--o{ Kontext : has
|
|
|
|
Parzelle ||--o{ GeoPolylinie : contains
|
|
Parzelle ||--o{ GeoPunkt : contains
|
|
Parzelle }o--|| Gemeinde : located_in
|
|
|
|
Land ||--o{ Kanton : contains
|
|
Kanton ||--o{ Gemeinde : contains
|
|
|
|
Land ||--o{ Dokument : has
|
|
Land ||--o{ Kontext : has
|
|
Kanton ||--o{ Dokument : has
|
|
Kanton ||--o{ Kontext : has
|
|
Gemeinde ||--o{ Dokument : has
|
|
Gemeinde ||--o{ Kontext : has
|
|
|
|
GeoPolylinie ||--o{ GeoPunkt : contains
|
|
```
|
|
|
|
### datamodelChat.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
ChatWorkflow {
|
|
string id PK
|
|
string mandateId
|
|
string status
|
|
string name
|
|
int currentRound
|
|
int currentTask
|
|
int currentAction
|
|
int totalTasks
|
|
int totalActions
|
|
float lastActivity
|
|
float startedAt
|
|
string workflowMode
|
|
int maxSteps
|
|
json logs
|
|
json messages
|
|
json stats
|
|
json tasks
|
|
}
|
|
|
|
ChatMessage {
|
|
string id PK
|
|
string workflowId FK
|
|
string parentMessageId FK
|
|
string message
|
|
string summary
|
|
string role
|
|
string status
|
|
int sequenceNr
|
|
float publishedAt
|
|
bool success
|
|
string actionId
|
|
json documents
|
|
}
|
|
|
|
ChatLog {
|
|
string id PK
|
|
string workflowId FK
|
|
string message
|
|
string type
|
|
float timestamp
|
|
string status
|
|
float progress
|
|
json performance
|
|
}
|
|
|
|
ChatStat {
|
|
string id PK
|
|
string workflowId FK
|
|
float processingTime
|
|
int bytesSent
|
|
int bytesReceived
|
|
int errorCount
|
|
string process
|
|
string engine
|
|
float priceUsd
|
|
}
|
|
|
|
ChatDocument {
|
|
string id PK
|
|
string messageId FK
|
|
string fileId FK
|
|
string fileName
|
|
int fileSize
|
|
string mimeType
|
|
int roundNumber
|
|
int taskNumber
|
|
int actionNumber
|
|
string actionId
|
|
}
|
|
|
|
TaskPlan {
|
|
string overview
|
|
json tasks
|
|
string userMessage
|
|
}
|
|
|
|
TaskItem {
|
|
string id PK
|
|
string workflowId FK
|
|
string userInput
|
|
string status
|
|
string error
|
|
float startedAt
|
|
float finishedAt
|
|
json actionList
|
|
int retryCount
|
|
int retryMax
|
|
bool rollbackOnFailure
|
|
json dependencies
|
|
string feedback
|
|
float processingTime
|
|
json resultLabels
|
|
}
|
|
|
|
TaskStep {
|
|
string id PK
|
|
string objective
|
|
json dependencies
|
|
json successCriteria
|
|
string estimatedComplexity
|
|
string userMessage
|
|
string dataType
|
|
json expectedFormats
|
|
json qualityRequirements
|
|
}
|
|
|
|
ActionItem {
|
|
string id PK
|
|
string execMethod
|
|
string execAction
|
|
json execParameters
|
|
string execResultLabel
|
|
json expectedDocumentFormats
|
|
string userMessage
|
|
string status
|
|
string error
|
|
int retryCount
|
|
int retryMax
|
|
float processingTime
|
|
float timestamp
|
|
string result
|
|
}
|
|
|
|
ActionResult {
|
|
bool success
|
|
string error
|
|
json documents
|
|
string resultLabel
|
|
}
|
|
|
|
AutomationDefinition {
|
|
string id PK
|
|
string mandateId
|
|
string label
|
|
string schedule
|
|
string template
|
|
json placeholders
|
|
bool active
|
|
string eventId
|
|
string status
|
|
json executionLogs
|
|
}
|
|
|
|
ChatWorkflow ||--o{ ChatMessage : contains
|
|
ChatWorkflow ||--o{ ChatLog : contains
|
|
ChatWorkflow ||--o{ ChatStat : contains
|
|
ChatWorkflow ||--o{ TaskPlan : has
|
|
ChatWorkflow ||--o{ AutomationDefinition : defines
|
|
|
|
ChatMessage ||--o{ ChatDocument : references
|
|
TaskPlan ||--o{ TaskStep : contains
|
|
TaskItem ||--o{ ActionItem : contains
|
|
ActionItem ||--o{ ActionResult : produces
|
|
```
|
|
|
|
### datamodelAi.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
AiModel {
|
|
string name PK
|
|
string displayName
|
|
string connectorType
|
|
string apiUrl
|
|
float temperature
|
|
int maxTokens
|
|
int contextLength
|
|
float costPer1kTokensInput
|
|
float costPer1kTokensOutput
|
|
int speedRating
|
|
int qualityRating
|
|
string priority
|
|
string processingMode
|
|
json operationTypes
|
|
int minContextLength
|
|
bool isAvailable
|
|
string version
|
|
string lastUpdated
|
|
}
|
|
|
|
OperationTypeRating {
|
|
string operationType
|
|
int rating
|
|
}
|
|
|
|
AiCallOptions {
|
|
string operationType
|
|
string priority
|
|
bool compressPrompt
|
|
bool compressContext
|
|
bool processDocumentsIndividually
|
|
float maxCost
|
|
int maxProcessingTime
|
|
string processingMode
|
|
string resultFormat
|
|
float safetyMargin
|
|
float temperature
|
|
int maxParts
|
|
}
|
|
|
|
AiCallRequest {
|
|
string prompt
|
|
string context
|
|
json options
|
|
json contentParts
|
|
}
|
|
|
|
AiCallResponse {
|
|
string content
|
|
string modelName
|
|
float priceUsd
|
|
float processingTime
|
|
int bytesSent
|
|
int bytesReceived
|
|
int errorCount
|
|
}
|
|
|
|
AiModelCall {
|
|
json messages
|
|
json model
|
|
json options
|
|
}
|
|
|
|
AiModelResponse {
|
|
string content
|
|
bool success
|
|
string error
|
|
string modelId
|
|
float processingTime
|
|
json tokensUsed
|
|
json metadata
|
|
}
|
|
|
|
AiModel ||--o{ OperationTypeRating : has
|
|
AiCallRequest ||--|| AiCallOptions : uses
|
|
AiCallRequest ||--|| AiCallResponse : produces
|
|
AiCallRequest }o--|| AiModel : uses
|
|
AiModelCall }o--|| AiModel : uses
|
|
AiModelCall ||--|| AiCallOptions : uses
|
|
AiModelCall ||--|| AiModelResponse : produces
|
|
```
|
|
|
|
### datamodelUam.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
Mandate {
|
|
string id PK
|
|
string name
|
|
string language
|
|
bool enabled
|
|
}
|
|
|
|
User {
|
|
string id PK
|
|
string username
|
|
string email
|
|
string fullName
|
|
string language
|
|
bool enabled
|
|
string privilege
|
|
string authenticationAuthority
|
|
string mandateId FK
|
|
}
|
|
|
|
UserConnection {
|
|
string id PK
|
|
string userId FK
|
|
string authority
|
|
string externalId
|
|
string externalUsername
|
|
string externalEmail
|
|
string status
|
|
float connectedAt
|
|
float lastChecked
|
|
float expiresAt
|
|
string tokenStatus
|
|
float tokenExpiresAt
|
|
}
|
|
|
|
UserInDB {
|
|
string id PK
|
|
string username
|
|
string email
|
|
string fullName
|
|
string language
|
|
bool enabled
|
|
string privilege
|
|
string authenticationAuthority
|
|
string mandateId FK
|
|
string hashedPassword
|
|
}
|
|
|
|
Mandate ||--o{ User : contains
|
|
User ||--o{ UserConnection : has
|
|
User ||--|| UserInDB : extends
|
|
```
|
|
|
|
### datamodelSecurity.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
Token {
|
|
string id PK
|
|
string userId FK
|
|
string authority
|
|
string connectionId FK
|
|
string tokenAccess
|
|
string tokenType
|
|
float expiresAt
|
|
string tokenRefresh
|
|
float createdAt
|
|
string status
|
|
float revokedAt
|
|
string revokedBy
|
|
string reason
|
|
string sessionId
|
|
string mandateId FK
|
|
}
|
|
|
|
AuthEvent {
|
|
string id PK
|
|
string userId FK
|
|
string eventType
|
|
float timestamp
|
|
string ipAddress
|
|
string userAgent
|
|
bool success
|
|
string details
|
|
}
|
|
|
|
Token ||--o{ AuthEvent : generates
|
|
```
|
|
|
|
### datamodelFiles.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
FileItem {
|
|
string id PK
|
|
string mandateId FK
|
|
string fileName
|
|
string mimeType
|
|
string fileHash
|
|
int fileSize
|
|
float creationDate
|
|
}
|
|
|
|
FilePreview {
|
|
string content
|
|
string mimeType
|
|
string fileName
|
|
bool isText
|
|
string encoding
|
|
int size
|
|
}
|
|
|
|
FileData {
|
|
string id PK
|
|
string data
|
|
bool base64Encoded
|
|
}
|
|
|
|
FileItem ||--|| FilePreview : generates
|
|
FileItem ||--|| FileData : contains
|
|
```
|
|
|
|
### datamodelDocument.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
StructuredDocument {
|
|
json metadata
|
|
json sections
|
|
string summary
|
|
json tags
|
|
}
|
|
|
|
DocumentMetadata {
|
|
string title
|
|
string author
|
|
datetime createdAt
|
|
json sourceDocuments
|
|
string extractionMethod
|
|
string version
|
|
}
|
|
|
|
DocumentSection {
|
|
string id PK
|
|
string title
|
|
string contentType
|
|
json elements
|
|
int order
|
|
json metadata
|
|
}
|
|
|
|
Paragraph {
|
|
string text
|
|
json formatting
|
|
json metadata
|
|
}
|
|
|
|
Heading {
|
|
string text
|
|
int level
|
|
json metadata
|
|
}
|
|
|
|
CodeBlock {
|
|
string code
|
|
string language
|
|
json metadata
|
|
}
|
|
|
|
Image {
|
|
string data
|
|
string altText
|
|
string caption
|
|
json metadata
|
|
}
|
|
|
|
BulletList {
|
|
json items
|
|
string listType
|
|
json metadata
|
|
}
|
|
|
|
ListItem {
|
|
string text
|
|
json subitems
|
|
json metadata
|
|
}
|
|
|
|
TableData {
|
|
json headers
|
|
json rows
|
|
string caption
|
|
json metadata
|
|
}
|
|
|
|
StructuredDocument ||--|| DocumentMetadata : has
|
|
StructuredDocument ||--o{ DocumentSection : contains
|
|
DocumentSection ||--o{ Paragraph : can_contain
|
|
DocumentSection ||--o{ Heading : can_contain
|
|
DocumentSection ||--o{ CodeBlock : can_contain
|
|
DocumentSection ||--o{ Image : can_contain
|
|
DocumentSection ||--o{ BulletList : can_contain
|
|
DocumentSection ||--o{ TableData : can_contain
|
|
BulletList ||--o{ ListItem : contains
|
|
ListItem ||--o{ ListItem : contains
|
|
```
|
|
|
|
### datamodelExtraction.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
ContentExtracted {
|
|
string id PK
|
|
json parts
|
|
json summary
|
|
}
|
|
|
|
ContentPart {
|
|
string id PK
|
|
string parentId FK
|
|
string label
|
|
string typeGroup
|
|
string mimeType
|
|
string data
|
|
json metadata
|
|
}
|
|
|
|
ExtractionOptions {
|
|
string prompt
|
|
string operationType
|
|
bool processDocumentsIndividually
|
|
int imageMaxPixels
|
|
int imageQuality
|
|
json mergeStrategy
|
|
bool chunkAllowed
|
|
int maxSize
|
|
int textChunkSize
|
|
int imageChunkSize
|
|
bool enableParallelProcessing
|
|
int maxConcurrentChunks
|
|
}
|
|
|
|
MergeStrategy {
|
|
string groupBy
|
|
string orderBy
|
|
string mergeType
|
|
int maxSize
|
|
json textMerge
|
|
json tableMerge
|
|
json structureMerge
|
|
json aiResultMerge
|
|
bool preserveChunks
|
|
string chunkSeparator
|
|
bool preserveMetadata
|
|
json metadataFields
|
|
string onError
|
|
bool validateContent
|
|
bool useIntelligentMerging
|
|
string prompt
|
|
json capabilities
|
|
}
|
|
|
|
PartResult {
|
|
json originalPart
|
|
string aiResult
|
|
int partIndex
|
|
string documentId
|
|
float processingTime
|
|
json metadata
|
|
}
|
|
|
|
ChunkResult {
|
|
json originalChunk
|
|
string aiResult
|
|
int chunkIndex
|
|
string documentId
|
|
float processingTime
|
|
json metadata
|
|
}
|
|
|
|
ContentExtracted ||--o{ ContentPart : contains
|
|
ContentPart ||--o{ ContentPart : parent_of
|
|
ContentExtracted ||--|| ExtractionOptions : uses
|
|
ExtractionOptions ||--|| MergeStrategy : uses
|
|
ContentPart ||--|| PartResult : produces
|
|
ContentPart ||--|| ChunkResult : produces
|
|
```
|
|
|
|
### datamodelPagination.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
PaginationParams {
|
|
int page
|
|
int pageSize
|
|
string sortBy
|
|
string sortOrder
|
|
}
|
|
|
|
PaginationRequest {
|
|
json params
|
|
json sortFields
|
|
}
|
|
|
|
SortField {
|
|
string field
|
|
string order
|
|
}
|
|
|
|
PaginatedResult {
|
|
json items
|
|
json metadata
|
|
json params
|
|
}
|
|
|
|
PaginationMetadata {
|
|
int page
|
|
int pageSize
|
|
int totalItems
|
|
int totalPages
|
|
bool hasNext
|
|
bool hasPrevious
|
|
}
|
|
|
|
PaginationRequest ||--|| PaginationParams : uses
|
|
PaginationRequest ||--o{ SortField : contains
|
|
PaginatedResult ||--|| PaginationMetadata : has
|
|
PaginatedResult ||--o{ PaginationParams : uses
|
|
```
|
|
|
|
### datamodelVoice.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
VoiceSettings {
|
|
string id PK
|
|
string userId FK
|
|
string language
|
|
string voice
|
|
json settings
|
|
}
|
|
```
|
|
|
|
### datamodelTickets.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
TicketFieldAttribute {
|
|
string fieldName PK
|
|
string fieldType
|
|
json fieldConfig
|
|
}
|
|
```
|
|
|
|
### datamodelNeutralizer.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
DataNeutraliserConfig {
|
|
string id PK
|
|
string mandateId FK
|
|
string name
|
|
bool enabled
|
|
json attributes
|
|
}
|
|
|
|
DataNeutralizerAttributes {
|
|
string fieldName PK
|
|
string neutralizationType
|
|
json options
|
|
}
|
|
|
|
DataNeutraliserConfig ||--o{ DataNeutralizerAttributes : contains
|
|
```
|
|
|
|
### datamodelUtils.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
Prompt {
|
|
string id PK
|
|
string name
|
|
string content
|
|
json metadata
|
|
}
|
|
```
|
|
|
|
### datamodelTools.py
|
|
|
|
```mermaid
|
|
erDiagram
|
|
CountryCodes {
|
|
string ISO2Code PK
|
|
string tavilyName
|
|
string perplexityName
|
|
}
|
|
```
|
|
|
|
**Note**: `CountryCodes` is a utility class (not a Pydantic BaseModel) that provides static methods for country code mapping. It contains a mapping dictionary but is not persisted to the database.
|
|
|
|
### datamodelJson.py
|
|
|
|
No database models - contains JSON template constants and supported section types.
|
|
|
|
---
|
|
|
|
## Interfaces Component
|
|
|
|
### Overview
|
|
|
|
The Interfaces component provides a clean abstraction layer for data access operations. Interfaces handle CRUD operations, user context management, access control, and integration with database connectors and external systems.
|
|
|
|
### Objects vs Access Files
|
|
|
|
Interfaces are split into two file types:
|
|
|
|
#### Objects Files (`interface*Objects.py`)
|
|
|
|
**Purpose**: Business logic and CRUD operations for data entities.
|
|
|
|
**Responsibilities**:
|
|
- CRUD operations (Create, Read, Update, Delete)
|
|
- Data validation and transformation
|
|
- Business rule enforcement
|
|
- Database/external system communication
|
|
- User context management
|
|
- Pagination and filtering
|
|
|
|
**Pattern**: Each Objects file contains methods for manipulating domain entities (e.g., `createProjekt()`, `getWorkflow()`, `updateUser()`).
|
|
|
|
#### Access Files (`interface*Access.py`)
|
|
|
|
**Purpose**: Permission checking and data filtering based on user privileges.
|
|
|
|
**Responsibilities**:
|
|
- User privilege validation
|
|
- Mandate-based filtering
|
|
- Record ownership checking
|
|
- Access control attribute generation (`_hideView`, `_hideEdit`, `_hideDelete`)
|
|
- Permission decision logic
|
|
|
|
**Pattern**: Access files contain two main methods:
|
|
- `uam()`: Unified Access Management - filters recordsets and adds access control flags
|
|
- `canModify()`: Checks if user can create/update/delete records
|
|
|
|
**Relationship**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Objects[Objects File] --> Access[Access File]
|
|
Access --> UAM[uam Method]
|
|
Access --> CanModify[canModify Method]
|
|
|
|
Objects --> CRUD[CRUD Operations]
|
|
CRUD --> AccessCheck{Check Access}
|
|
AccessCheck -->|Read| UAM
|
|
AccessCheck -->|Write| CanModify
|
|
|
|
UAM --> Filter[Filter Records]
|
|
UAM --> Flags[Add Access Flags]
|
|
|
|
CanModify --> Permission{Permission?}
|
|
Permission -->|Yes| Allow[Allow Operation]
|
|
Permission -->|No| Deny[Deny Operation]
|
|
```
|
|
|
|
### Interface Types
|
|
|
|
Interfaces are categorized into two types based on their data source:
|
|
|
|
#### Database Interfaces
|
|
|
|
**Why Database Connectors?**: These interfaces manage persistent data stored in PostgreSQL databases. They use database connectors to:
|
|
- Store structured data with relationships
|
|
- Ensure data consistency and integrity
|
|
- Provide ACID transactions
|
|
- Support complex queries and filtering
|
|
- Enable mandate-based data isolation
|
|
|
|
**Characteristics**:
|
|
- Use `DatabaseConnector` for PostgreSQL access
|
|
- Implement Access classes for permission control
|
|
- Support pagination and sorting
|
|
- Apply mandate-based filtering automatically
|
|
- Track record ownership (`_createdBy`)
|
|
|
|
#### External System Interfaces
|
|
|
|
**Why External Connectors?**: These interfaces integrate with external APIs and services. They use connectors to:
|
|
- Communicate with third-party systems
|
|
- Transform data between formats
|
|
- Handle API authentication and rate limiting
|
|
- Provide abstraction over external service complexity
|
|
|
|
**Characteristics**:
|
|
- Use specialized connectors (e.g., `ConnectorGoogleSpeech`, `ConnectorTicketJira`)
|
|
- May not require user context (system-level operations)
|
|
- Focus on data transformation and synchronization
|
|
- Handle external API errors and retries
|
|
|
|
### Real Estate Interface (`interfaceDbRealEstateObjects.py`)
|
|
|
|
**Type**: Database Interface
|
|
**Database**: PostgreSQL (Real Estate database)
|
|
**Access Control**: `interfaceDbRealEstateAccess.py` → `RealEstateAccess`
|
|
|
|
**Purpose**: Manages real estate domain data including projects, parcels, administrative entities, and geographic information.
|
|
|
|
**Why Database Connector**: Real estate data requires persistent storage with complex relationships (projects → parcels → administrative units), geographic data (polygons, points), and mandate-based isolation for multi-tenant scenarios.
|
|
|
|
**CRUD Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Projekt Operations"
|
|
PCreate[createProjekt]
|
|
PGet[getProjekt]
|
|
PGetAll[getProjekte]
|
|
PUpdate[updateProjekt]
|
|
PDelete[deleteProjekt]
|
|
end
|
|
|
|
subgraph "Parzelle Operations"
|
|
ParCreate[createParzelle]
|
|
ParGet[getParzelle]
|
|
ParGetAll[getParzellen]
|
|
ParUpdate[updateParzelle]
|
|
ParDelete[deleteParzelle]
|
|
end
|
|
|
|
subgraph "Dokument Operations"
|
|
DocCreate[createDokument]
|
|
DocGet[getDokument]
|
|
DocGetAll[getDokumente]
|
|
DocUpdate[updateDokument]
|
|
DocDelete[deleteDokument]
|
|
end
|
|
|
|
subgraph "Administrative Hierarchy"
|
|
GemCreate[createGemeinde]
|
|
GemGet[getGemeinde]
|
|
GemGetAll[getGemeinden]
|
|
GemUpdate[updateGemeinde]
|
|
GemDelete[deleteGemeinde]
|
|
|
|
KanCreate[createKanton]
|
|
KanGet[getKanton]
|
|
KanGetAll[getKantone]
|
|
KanUpdate[updateKanton]
|
|
KanDelete[deleteKanton]
|
|
|
|
LanCreate[createLand]
|
|
LanGet[getLand]
|
|
LanGetAll[getLaender]
|
|
LanUpdate[updateLand]
|
|
LanDelete[deleteLand]
|
|
end
|
|
|
|
subgraph "Kontext Operations"
|
|
KonCreate[createKontext]
|
|
KonGet[getKontext]
|
|
KonGetAll[getKontexte]
|
|
KonUpdate[updateKontext]
|
|
KonDelete[deleteKontext]
|
|
end
|
|
```
|
|
|
|
**Complete CRUD List**:
|
|
|
|
**Projekt**:
|
|
- `createProjekt(projekt: Projekt) → Projekt`
|
|
- `getProjekt(projektId: str) → Optional[Projekt]`
|
|
- `getProjekte(recordFilter: Optional[Dict]) → List[Projekt]`
|
|
- `updateProjekt(projektId: str, updateData: Dict) → Optional[Projekt]`
|
|
- `deleteProjekt(projektId: str) → bool`
|
|
|
|
**Parzelle**:
|
|
- `createParzelle(parzelle: Parzelle) → Parzelle`
|
|
- `getParzelle(parzelleId: str) → Optional[Parzelle]`
|
|
- `getParzellen(recordFilter: Optional[Dict]) → List[Parzelle]`
|
|
- `updateParzelle(parzelleId: str, updateData: Dict) → Optional[Parzelle]`
|
|
- `deleteParzelle(parzelleId: str) → bool`
|
|
|
|
**Dokument**:
|
|
- `createDokument(dokument: Dokument) → Dokument`
|
|
- `getDokument(dokumentId: str) → Optional[Dokument]`
|
|
- `getDokumente(recordFilter: Optional[Dict]) → List[Dokument]`
|
|
- `updateDokument(dokumentId: str, updateData: Dict) → Optional[Dokument]`
|
|
- `deleteDokument(dokumentId: str) → bool`
|
|
|
|
**Gemeinde**:
|
|
- `createGemeinde(gemeinde: Gemeinde) → Gemeinde`
|
|
- `getGemeinde(gemeindeId: str) → Optional[Gemeinde]`
|
|
- `getGemeinden(recordFilter: Optional[Dict]) → List[Gemeinde]`
|
|
- `updateGemeinde(gemeindeId: str, updateData: Dict) → Optional[Gemeinde]`
|
|
- `deleteGemeinde(gemeindeId: str) → bool`
|
|
|
|
**Kanton**:
|
|
- `createKanton(kanton: Kanton) → Kanton`
|
|
- `getKanton(kantonId: str) → Optional[Kanton]`
|
|
- `getKantone(recordFilter: Optional[Dict]) → List[Kanton]`
|
|
- `updateKanton(kantonId: str, updateData: Dict) → Optional[Kanton]`
|
|
- `deleteKanton(kantonId: str) → bool`
|
|
|
|
**Land**:
|
|
- `createLand(land: Land) → Land`
|
|
- `getLand(landId: str) → Optional[Land]`
|
|
- `getLaender(recordFilter: Optional[Dict]) → List[Land]`
|
|
- `updateLand(landId: str, updateData: Dict) → Optional[Land]`
|
|
- `deleteLand(landId: str) → bool`
|
|
|
|
**Kontext**:
|
|
- `createKontext(kontext: Kontext) → Kontext`
|
|
- `getKontext(kontextId: str) → Optional[Kontext]`
|
|
- `getKontexte(recordFilter: Optional[Dict]) → List[Kontext]`
|
|
- `updateKontext(kontextId: str, updateData: Dict) → Optional[Kontext]`
|
|
- `deleteKontext(kontextId: str) → bool`
|
|
|
|
**Access Control Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Request[CRUD Request] --> Objects[RealEstateObjects]
|
|
Objects --> Access[RealEstateAccess]
|
|
|
|
Access --> CheckPriv[Check Privilege]
|
|
CheckPriv -->|SYSADMIN| AllData[All Records]
|
|
CheckPriv -->|ADMIN| MandateData[Mandate Records]
|
|
CheckPriv -->|USER| OwnData[Own Records]
|
|
|
|
AllData --> UAM[uam Method]
|
|
MandateData --> UAM
|
|
OwnData --> UAM
|
|
|
|
UAM --> Filter[Filter by mandateId]
|
|
UAM --> CheckOwn[Check _createdBy]
|
|
UAM --> AddFlags[Add _hideView/_hideEdit/_hideDelete]
|
|
|
|
Filter --> Return[Return Filtered Data]
|
|
CheckOwn --> Return
|
|
AddFlags --> Return
|
|
```
|
|
|
|
**Key Features**:
|
|
- Geographic data support (GeoPolylinie, GeoPunkt)
|
|
- Multi-level administrative hierarchy (Land → Kanton → Gemeinde)
|
|
- Location name resolution (converts names to IDs for filtering)
|
|
- Document versioning and management
|
|
- Context information for projects and administrative units
|
|
|
|
### Chat Interface (`interfaceDbChatObjects.py`)
|
|
|
|
**Type**: Database Interface
|
|
**Database**: PostgreSQL (Chat database)
|
|
**Access Control**: `interfaceDbChatAccess.py` → `ChatAccess`
|
|
|
|
**Purpose**: Manages chat workflows, messages, logs, statistics, and automation definitions for AI-powered conversation workflows.
|
|
|
|
**Why Database Connector**: Chat workflows require persistent storage for conversation history, workflow state, performance metrics, and automation configurations. Data must be queryable, filterable, and mandate-isolated.
|
|
|
|
**CRUD Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "ChatWorkflow Operations"
|
|
WfGet[getWorkflows - with pagination]
|
|
WfGetOne[getWorkflow - by ID]
|
|
WfCreate[createWorkflow]
|
|
WfUpdate[updateWorkflow]
|
|
WfDelete[deleteWorkflow - cascade]
|
|
end
|
|
|
|
subgraph "ChatMessage Operations"
|
|
MsgGet[getMessages - by workflowId, pagination]
|
|
MsgCreate[createMessage]
|
|
MsgUpdate[updateMessage]
|
|
MsgDelete[deleteMessage]
|
|
MsgDeleteFile[deleteFileFromMessage]
|
|
end
|
|
|
|
subgraph "ChatDocument Operations"
|
|
DocGet[getDocuments - by messageId]
|
|
DocCreate[createDocument]
|
|
end
|
|
|
|
subgraph "ChatLog Operations"
|
|
LogGet[getLogs - by workflowId, pagination]
|
|
LogCreate[createLog]
|
|
end
|
|
|
|
subgraph "ChatStat Operations"
|
|
StatGet[getStats - by workflowId]
|
|
StatCreate[createStat]
|
|
end
|
|
|
|
subgraph "AutomationDefinition Operations"
|
|
AutoGet[getAllAutomationDefinitions - pagination]
|
|
AutoGetOne[getAutomationDefinition - by ID]
|
|
AutoCreate[createAutomationDefinition]
|
|
AutoUpdate[updateAutomationDefinition]
|
|
AutoDelete[deleteAutomationDefinition]
|
|
end
|
|
|
|
subgraph "Utility Operations"
|
|
Unified[getUnifiedChatData - workflow snapshot]
|
|
end
|
|
```
|
|
|
|
**Complete CRUD List**:
|
|
|
|
**ChatWorkflow**:
|
|
- `getWorkflows(pagination: Optional[PaginationParams]) → Union[List[Dict], PaginatedResult]`
|
|
- `getWorkflow(workflowId: str) → Optional[ChatWorkflow]`
|
|
- `createWorkflow(workflowData: Dict) → ChatWorkflow`
|
|
- `updateWorkflow(workflowId: str, workflowData: Dict) → ChatWorkflow`
|
|
- `deleteWorkflow(workflowId: str) → bool` (cascades to messages, logs, stats)
|
|
|
|
**ChatMessage**:
|
|
- `getMessages(workflowId: str, pagination: Optional[PaginationParams]) → Union[List[ChatMessage], PaginatedResult]`
|
|
- `createMessage(messageData: Dict) → ChatMessage`
|
|
- `updateMessage(messageId: str, messageData: Dict) → Dict`
|
|
- `deleteMessage(workflowId: str, messageId: str) → bool`
|
|
- `deleteFileFromMessage(workflowId: str, messageId: str, fileId: str) → bool`
|
|
|
|
**ChatDocument**:
|
|
- `getDocuments(messageId: str) → List[ChatDocument]`
|
|
- `createDocument(documentData: Dict) → ChatDocument`
|
|
|
|
**ChatLog**:
|
|
- `getLogs(workflowId: str, pagination: Optional[PaginationParams]) → Union[List[ChatLog], PaginatedResult]`
|
|
- `createLog(logData: Dict) → ChatLog`
|
|
|
|
**ChatStat**:
|
|
- `getStats(workflowId: str) → List[ChatStat]`
|
|
- `createStat(statData: Dict) → ChatStat`
|
|
|
|
**AutomationDefinition**:
|
|
- `getAllAutomationDefinitions(pagination: Optional[PaginationParams]) → Union[List[Dict], PaginatedResult]`
|
|
- `getAutomationDefinition(automationId: str) → Optional[Dict]`
|
|
- `createAutomationDefinition(automationData: Dict) → Dict`
|
|
- `updateAutomationDefinition(automationId: str, automationData: Dict) → Dict`
|
|
- `deleteAutomationDefinition(automationId: str) → bool`
|
|
|
|
**Utility Methods**:
|
|
- `getUnifiedChatData(workflowId: str, afterTimestamp: Optional[float]) → Dict` (returns workflow snapshot with messages, logs, stats)
|
|
|
|
**Access Control Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Request[CRUD Request] --> Objects[ChatObjects]
|
|
Objects --> Access[ChatAccess]
|
|
|
|
Access --> CheckPriv[Check Privilege]
|
|
CheckPriv -->|SYSADMIN| AllWorkflows[All Workflows]
|
|
CheckPriv -->|ADMIN| MandateWorkflows[Mandate Workflows]
|
|
CheckPriv -->|USER| OwnWorkflows[Own Workflows]
|
|
|
|
AllWorkflows --> UAM[uam Method]
|
|
MandateWorkflows --> UAM
|
|
OwnWorkflows --> UAM
|
|
|
|
UAM --> FilterWorkflow[Filter by workflowId mandate]
|
|
UAM --> CheckOwn[Check _createdBy]
|
|
UAM --> AddFlags[Add access flags]
|
|
|
|
FilterWorkflow --> CheckChild[Check Child Access]
|
|
CheckChild -->|Message| CheckWorkflow[Check workflow ownership]
|
|
CheckChild -->|Log| CheckWorkflow
|
|
CheckChild -->|Stat| CheckWorkflow
|
|
|
|
CheckWorkflow --> Return[Return Filtered Data]
|
|
CheckOwn --> Return
|
|
AddFlags --> Return
|
|
```
|
|
|
|
**Key Features**:
|
|
- Multi-round workflow support with state tracking
|
|
- Normalized data model (workflows, messages, logs, stats in separate tables)
|
|
- Cascade delete (deleting workflow removes all related data)
|
|
- Pagination support for large datasets
|
|
- Unified data retrieval for workflow snapshots
|
|
- Automation workflow definitions
|
|
- Document attachment management
|
|
|
|
### App Interface (`interfaceDbAppObjects.py`)
|
|
|
|
**Type**: Database Interface
|
|
**Database**: PostgreSQL (App database)
|
|
**Access Control**: `interfaceDbAppAccess.py` → `AppAccess`
|
|
|
|
**Purpose**: Manages users, mandates, authentication tokens, and application-level configuration.
|
|
|
|
**Why Database Connector**: User accounts, mandates, and authentication data require secure, persistent storage with strict access control. This is the foundation for all other interfaces' user context.
|
|
|
|
**CRUD Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "User Operations"
|
|
UserGet[getUsersByMandate - pagination]
|
|
UserGetByUsername[getUserByUsername]
|
|
UserGetOne[getUser - by ID]
|
|
UserCreate[createUser - with password hash]
|
|
UserUpdate[updateUser]
|
|
UserDelete[deleteUser]
|
|
end
|
|
|
|
subgraph "UserConnection Operations"
|
|
ConnGet[getUserConnections - by userId]
|
|
ConnGetToken[getConnectionToken - by connectionId]
|
|
end
|
|
|
|
subgraph "Mandate Operations"
|
|
ManGet[getAllMandates - pagination]
|
|
ManGetOne[getMandate - by ID]
|
|
ManCreate[createMandate]
|
|
ManUpdate[updateMandate]
|
|
ManDelete[deleteMandate]
|
|
end
|
|
|
|
subgraph "Neutralization Config"
|
|
NeuGet[getNeutralizationConfig]
|
|
NeuCreate[createOrUpdateNeutralizationConfig]
|
|
NeuGetAttrs[getNeutralizationAttributes]
|
|
NeuDeleteAttrs[deleteNeutralizationAttributes]
|
|
end
|
|
|
|
subgraph "Initialization"
|
|
InitRoot[getRootInterface - system init]
|
|
InitRecords[_initRootMandate, _initAdminUser, _initEventUser]
|
|
end
|
|
```
|
|
|
|
**Complete CRUD List**:
|
|
|
|
**User**:
|
|
- `getUsersByMandate(mandateId: str, pagination: Optional[PaginationParams]) → Union[List[User], PaginatedResult]`
|
|
- `getUserByUsername(username: str) → Optional[User]`
|
|
- `getUser(userId: str) → Optional[User]`
|
|
- `createUser(userData: Dict, password: Optional[str]) → User` (hashes password with Argon2)
|
|
- `updateUser(userId: str, updateData: Union[Dict, User]) → User`
|
|
- `deleteUser(userId: str) → bool`
|
|
|
|
**UserConnection**:
|
|
- `getUserConnections(userId: str) → List[UserConnection]`
|
|
- `getConnectionToken(connectionId: str) → Optional[Token]`
|
|
|
|
**Mandate**:
|
|
- `getAllMandates(pagination: Optional[PaginationParams]) → Union[List[Mandate], PaginatedResult]`
|
|
- `getMandate(mandateId: str) → Optional[Mandate]`
|
|
- `createMandate(name: str, language: str) → Mandate`
|
|
- `updateMandate(mandateId: str, updateData: Dict) → Mandate`
|
|
- `deleteMandate(mandateId: str) → bool`
|
|
|
|
**DataNeutraliserConfig**:
|
|
- `getNeutralizationConfig() → Optional[DataNeutraliserConfig]`
|
|
- `createOrUpdateNeutralizationConfig(configData: Dict) → DataNeutraliserConfig`
|
|
- `getNeutralizationAttributes(file_id: str) → List[Dict]`
|
|
- `deleteNeutralizationAttributes(file_id: str) → bool`
|
|
|
|
**Special Methods**:
|
|
- `getRootInterface() → AppObjects` (creates interface with system admin privileges for initialization)
|
|
- `getInitialId(model_class: type) → Optional[str]` (gets first record ID for a model)
|
|
|
|
**Access Control Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Request[CRUD Request] --> Objects[AppObjects]
|
|
Objects --> Access[AppAccess]
|
|
|
|
Access --> CheckPriv[Check Privilege]
|
|
CheckPriv -->|SYSADMIN| FullAccess[Full System Access]
|
|
CheckPriv -->|ADMIN| MandateAccess[Mandate Access Only]
|
|
CheckPriv -->|USER| SelfAccess[Own User Record Only]
|
|
|
|
FullAccess --> UserOps[User Operations]
|
|
MandateAccess --> UserOps
|
|
SelfAccess --> UserOps
|
|
|
|
UserOps --> CheckOwn[Check Ownership]
|
|
CheckOwn -->|User CRUD| SYSADMINOnly{SYSADMIN?}
|
|
CheckOwn -->|Mandate CRUD| SYSADMINOnly
|
|
CheckOwn -->|Own Profile| Allow[Allow]
|
|
|
|
SYSADMINOnly -->|Yes| Allow
|
|
SYSADMINOnly -->|No| Deny[Deny]
|
|
```
|
|
|
|
**Key Features**:
|
|
- Password hashing with Argon2
|
|
- Multi-provider authentication support (local, external)
|
|
- System initialization (Root mandate, Admin user, Event user)
|
|
- Mandate-based user isolation
|
|
- Token management for external connections
|
|
- Data neutralization configuration
|
|
|
|
### Component Interface (`interfaceDbComponentObjects.py`)
|
|
|
|
**Type**: Database Interface
|
|
**Database**: PostgreSQL (Component/Management database)
|
|
**Access Control**: `interfaceDbComponentAccess.py` → `ComponentAccess`
|
|
|
|
**Purpose**: Manages component-level data including files, prompts, and voice settings used across the application.
|
|
|
|
**Why Database Connector**: Files, prompts, and voice settings require persistent storage with metadata, ownership tracking, and mandate isolation. Files need binary storage with preview generation.
|
|
|
|
**CRUD Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "File Operations"
|
|
FileGetAll[getAllFiles - pagination]
|
|
FileGet[getFile - by ID]
|
|
FileCreate[createFile - name, mimeType, content]
|
|
FileUpdate[updateFile]
|
|
FileDelete[deleteFile]
|
|
FileGetData[getFileData - binary content]
|
|
FileGetContent[getFileContent - preview]
|
|
FileCreateData[createFileData - store binary]
|
|
end
|
|
|
|
subgraph "Prompt Operations"
|
|
PromptGetAll[getAllPrompts - pagination]
|
|
PromptGet[getPrompt - by ID]
|
|
PromptCreate[createPrompt]
|
|
PromptUpdate[updatePrompt]
|
|
PromptDelete[deletePrompt]
|
|
end
|
|
|
|
subgraph "VoiceSettings Operations"
|
|
VoiceGet[getVoiceSettings - by userId]
|
|
VoiceCreate[createVoiceSettings]
|
|
VoiceUpdate[updateVoiceSettings]
|
|
VoiceDelete[deleteVoiceSettings]
|
|
VoiceGetOrCreate[getOrCreateVoiceSettings]
|
|
end
|
|
|
|
subgraph "Utilities"
|
|
MimeType[getMimeType - from fileName]
|
|
end
|
|
```
|
|
|
|
**Complete CRUD List**:
|
|
|
|
**FileItem**:
|
|
- `getAllFiles(pagination: Optional[PaginationParams]) → Union[List[FileItem], PaginatedResult]`
|
|
- `getFile(fileId: str) → Optional[FileItem]`
|
|
- `createFile(name: str, mimeType: str, content: bytes) → FileItem` (creates FileItem and FileData)
|
|
- `updateFile(fileId: str, updateData: Dict) → Dict`
|
|
- `deleteFile(fileId: str) → bool` (deletes FileItem and FileData)
|
|
- `getFileData(fileId: str) → Optional[bytes]` (raw binary content)
|
|
- `getFileContent(fileId: str) → Optional[FilePreview]` (generates preview)
|
|
- `createFileData(fileId: str, data: bytes) → bool` (stores binary data)
|
|
|
|
**Prompt**:
|
|
- `getAllPrompts(pagination: Optional[PaginationParams]) → Union[List[Prompt], PaginatedResult]`
|
|
- `getPrompt(promptId: str) → Optional[Prompt]`
|
|
- `createPrompt(promptData: Dict) → Dict`
|
|
- `updatePrompt(promptId: str, updateData: Dict) → Dict`
|
|
- `deletePrompt(promptId: str) → bool`
|
|
|
|
**VoiceSettings**:
|
|
- `getVoiceSettings(userId: Optional[str]) → Optional[VoiceSettings]`
|
|
- `createVoiceSettings(settingsData: Dict) → Dict`
|
|
- `updateVoiceSettings(userId: str, updateData: Dict) → Dict`
|
|
- `deleteVoiceSettings(userId: str) → bool`
|
|
- `getOrCreateVoiceSettings(userId: Optional[str]) → VoiceSettings`
|
|
|
|
**Utility Methods**:
|
|
- `getMimeType(fileName: str) → str` (detects MIME type from extension)
|
|
|
|
**Access Control Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Request[CRUD Request] --> Objects[ComponentObjects]
|
|
Objects --> Access[ComponentAccess]
|
|
|
|
Access --> CheckPriv[Check Privilege]
|
|
CheckPriv -->|SYSADMIN| AllFiles[All Files]
|
|
CheckPriv -->|ADMIN| MandateFiles[Mandate Files]
|
|
CheckPriv -->|USER| OwnFiles[Own Files]
|
|
|
|
AllFiles --> UAM[uam Method]
|
|
MandateFiles --> UAM
|
|
OwnFiles --> UAM
|
|
|
|
UAM --> FilterMandate[Filter by mandateId]
|
|
UAM --> CheckOwn[Check _createdBy]
|
|
UAM --> AddFlags[Add access flags]
|
|
|
|
FilterMandate --> Return[Return Filtered Data]
|
|
CheckOwn --> Return
|
|
AddFlags --> Return
|
|
```
|
|
|
|
**Key Features**:
|
|
- Binary file storage with metadata
|
|
- Automatic preview generation (text, images, etc.)
|
|
- MIME type detection
|
|
- Prompt template management with initialization
|
|
- Voice settings per user
|
|
- File hash calculation for deduplication
|
|
|
|
### AI Interface (`interfaceAiObjects.py`)
|
|
|
|
**Type**: External System Interface
|
|
**Connectors**: Dynamic discovery via `modelRegistry`
|
|
**Access Control**: None (system-level operations)
|
|
|
|
**Purpose**: Provides centralized AI operations with dynamic model discovery, automatic model selection, and failover handling.
|
|
|
|
**Why External Connector**: AI operations require communication with external APIs (OpenAI, Anthropic, Perplexity, Tavily) and internal AI services. The interface abstracts model selection, handles API calls, manages failover, and tracks costs.
|
|
|
|
**Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "AI Call Operations"
|
|
Call[call - main entry point]
|
|
CallText[call with text/context]
|
|
CallParts[call with content parts]
|
|
end
|
|
|
|
subgraph "Model Information"
|
|
ModelInfo[getModelInfo - by displayName]
|
|
ModelsByTag[getModelsByTag - filter by tag]
|
|
end
|
|
|
|
subgraph "Internal Processing"
|
|
SelectModel[_selectModel - dynamic selection]
|
|
ProcessPart[_processContentPartWithFallback]
|
|
MergeResults[_mergePartResults]
|
|
CallWithModel[_callWithModel - execute API call]
|
|
end
|
|
```
|
|
|
|
**Complete Operations List**:
|
|
|
|
**AI Calls**:
|
|
- `call(request: AiCallRequest, progressCallback=None) → AiCallResponse` (main entry point, handles text/context or content parts)
|
|
- `getModelInfo(displayName: str) → Dict[str, Any]` (get model metadata)
|
|
- `getModelsByTag(tag: str) → List[str]` (filter models by tag)
|
|
|
|
**Internal Methods** (used by call):
|
|
- `_selectModel(prompt: str, context: str, options: AiCallOptions) → str` (selects best model)
|
|
- `_callWithTextContext(request: AiCallRequest) → AiCallResponse` (handles traditional text/context calls)
|
|
- `_callWithContentParts(request: AiCallRequest, progressCallback) → AiCallResponse` (handles content parts with chunking)
|
|
- `_processContentPartWithFallback(...) → AiCallResponse` (processes single part with failover)
|
|
- `_callWithModel(model, prompt, context, options) → AiCallResponse` (executes actual API call)
|
|
|
|
**Model Selection and Failover Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Request[AI Call Request] --> CheckType{Request Type?}
|
|
|
|
CheckType -->|Text/Context| TextPath[Text/Context Path]
|
|
CheckType -->|Content Parts| PartsPath[Content Parts Path]
|
|
|
|
TextPath --> GetFailover[Get Failover Model List]
|
|
PartsPath --> GetFailover
|
|
|
|
GetFailover --> SelectModel[Select Best Model]
|
|
SelectModel --> FilterOp[Filter by Operation Type]
|
|
FilterOp --> RateCap[Rate by Capabilities]
|
|
RateCap --> ApplyPriority[Apply Priority Rules]
|
|
ApplyPriority --> TryModel[Try Model Call]
|
|
|
|
TryModel --> Success{Success?}
|
|
Success -->|Yes| Return[Return Response]
|
|
Success -->|No| NextModel{More Models?}
|
|
|
|
NextModel -->|Yes| TryModel
|
|
NextModel -->|No| Error[Return Error Response]
|
|
|
|
PartsPath --> ProcessEach[Process Each Part]
|
|
ProcessEach --> Chunk[Model-Aware Chunking]
|
|
Chunk --> TryModel
|
|
ProcessEach --> Merge[Merge Results]
|
|
Merge --> Return
|
|
```
|
|
|
|
**Supported External Systems**:
|
|
- **OpenAI**: GPT models via `aicorePluginOpenai`
|
|
- **Anthropic**: Claude models via `aicorePluginAnthropic`
|
|
- **Perplexity**: Search-enabled models via `aicorePluginPerplexity`
|
|
- **Tavily**: Web search API via `aicorePluginTavily`
|
|
- **Internal**: Custom models via `aicorePluginInternal`
|
|
|
|
**Key Features**:
|
|
- Dynamic model discovery (auto-registers available connectors)
|
|
- Operation type-based model selection (e.g., IMAGE_ANALYSE, TEXT_GENERATION)
|
|
- Automatic failover (tries multiple models on failure)
|
|
- Model-aware chunking (respects model context limits)
|
|
- Content part processing (handles images, text, tables)
|
|
- Cost tracking (calculates USD cost per call)
|
|
- Progress callbacks for long-running operations
|
|
|
|
### Ticket Interface (`interfaceTicketObjects.py`)
|
|
|
|
**Type**: External System Interface
|
|
**Connectors**: `ConnectorTicketJira`, `ConnectorTicketClickup`
|
|
**Access Control**: Connector-level (API credentials)
|
|
|
|
**Purpose**: Synchronizes data with external ticket systems (Jira, ClickUp) by transforming tickets to/from list format for Excel-like operations.
|
|
|
|
**Why External Connector**: Ticket systems are external services with their own APIs. The interface provides bidirectional synchronization, field mapping, and data transformation between the application's list format and ticket system formats.
|
|
|
|
**Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Factory Method"
|
|
Factory[createTicketInterfaceByType - connectorType, params]
|
|
end
|
|
|
|
subgraph "Export Operations"
|
|
Export[exportTicketsAsList - read from external]
|
|
end
|
|
|
|
subgraph "Import Operations"
|
|
Import[importListToTickets - write to external]
|
|
end
|
|
|
|
subgraph "Internal Transformation"
|
|
Transform[_transformTicketRecords - field mapping]
|
|
Extract[_extractFieldValue - path-based extraction]
|
|
FormatDate[_formatDateForExcel - date formatting]
|
|
FilterEmpty[_filterEmptyRecords - validation]
|
|
end
|
|
```
|
|
|
|
**Complete Operations List**:
|
|
|
|
**Factory**:
|
|
- `createTicketInterfaceByType(taskSyncDefinition: dict, connectorType: str, connectorParams: dict) → TicketInterface` (creates interface with appropriate connector)
|
|
|
|
**Export**:
|
|
- `exportTicketsAsList() → list[dict]` (reads tickets from external system, transforms to list format)
|
|
|
|
**Import**:
|
|
- `importListToTickets(records: list[dict]) → None` (transforms list format, writes to external system)
|
|
|
|
**Internal Methods**:
|
|
- `_transformTicketRecords(tasks: list[dict], includePut: bool) → list[dict]` (transforms according to task_sync_definition)
|
|
- `_extractFieldValue(issue_data: dict, field_path: list[str], field_name: str) → Any` (extracts value using path)
|
|
- `_formatDateForExcel(date_value: Any) → Optional[str]` (formats dates for Excel compatibility)
|
|
- `_isDateField(field_name: str) → bool` (detects date fields)
|
|
- `_filterEmptyRecords(records: list[dict]) → list[dict]` (removes invalid records)
|
|
|
|
**Synchronization Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Export Flow"
|
|
ExportStart[exportTicketsAsList] --> ReadConn[Read from Connector]
|
|
ReadConn --> GetTasks[Get Tasks from API]
|
|
GetTasks --> Transform[Transform Fields]
|
|
Transform --> MapFields[Map via task_sync_definition]
|
|
MapFields --> FormatDates[Format Dates]
|
|
FormatDates --> Filter[Filter Empty Records]
|
|
Filter --> ReturnList[Return List Format]
|
|
end
|
|
|
|
subgraph "Import Flow"
|
|
ImportStart[importListToTickets] --> ReceiveList[Receive List Format]
|
|
ReceiveList --> ExtractFields[Extract Fields]
|
|
ExtractFields --> MapToTicket[Map to Ticket Format]
|
|
MapToTicket --> BuildUpdate[Build Update Payload]
|
|
BuildUpdate --> WriteConn[Write via Connector]
|
|
WriteConn --> UpdateAPI[Update External API]
|
|
end
|
|
|
|
subgraph "Field Mapping"
|
|
SyncDef[task_sync_definition] --> Direction[Direction: get/put]
|
|
SyncDef --> Path[Field Path: nested access]
|
|
Direction --> Transform
|
|
Path --> MapFields
|
|
Path --> MapToTicket
|
|
end
|
|
```
|
|
|
|
**Supported External Systems**:
|
|
- **Jira**: Via `ConnectorTicketJira` (uses Jira REST API)
|
|
- **ClickUp**: Via `ConnectorTicketClickup` (uses ClickUp API)
|
|
|
|
**Key Features**:
|
|
- Dynamic connector selection (Jira or ClickUp)
|
|
- Field mapping configuration (task_sync_definition maps fields bidirectionally)
|
|
- Path-based field extraction (supports nested JSON structures)
|
|
- Date format handling (converts various formats to Excel-compatible)
|
|
- Bidirectional sync (export to list, import from list)
|
|
- Empty record filtering (validates records have IDs)
|
|
|
|
### Voice Interface (`interfaceVoiceObjects.py`)
|
|
|
|
**Type**: External System Interface
|
|
**Connector**: `ConnectorGoogleSpeech`
|
|
**Access Control**: User context for settings (stored in Component database)
|
|
|
|
**Purpose**: Provides speech-to-text, text-to-speech, and translation services using Google Cloud APIs.
|
|
|
|
**Why External Connector**: Voice operations require Google Cloud Speech-to-Text, Text-to-Speech, and Translation APIs. The interface abstracts API complexity, handles audio format conversion, and manages user voice settings (stored in database via Component interface).
|
|
|
|
**Operations**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Speech-to-Text Operations"
|
|
STT[speechToText - audio to text]
|
|
STTTrans[speechToTranslatedText - audio to translated text]
|
|
end
|
|
|
|
subgraph "Text-to-Speech Operations"
|
|
TTS[textToSpeech - text to audio]
|
|
TTSTrans[textToTranslatedSpeech - text to translated audio]
|
|
end
|
|
|
|
subgraph "Translation Operations"
|
|
Trans[translateText - text translation]
|
|
end
|
|
|
|
subgraph "Voice Settings"
|
|
GetSettings[getVoiceSettings - from Component DB]
|
|
CreateSettings[createVoiceSettings - to Component DB]
|
|
UpdateSettings[updateVoiceSettings - in Component DB]
|
|
GetOrCreate[getOrCreateVoiceSettings]
|
|
end
|
|
|
|
subgraph "Metadata Operations"
|
|
GetLangs[getAvailableLanguages - from Google API]
|
|
GetVoices[getAvailableVoices - from Google API]
|
|
end
|
|
```
|
|
|
|
**Complete Operations List**:
|
|
|
|
**Speech-to-Text**:
|
|
- `speechToText(audioContent: bytes, language: str, sampleRate: Optional[int], channels: Optional[int]) → Dict[str, Any]` (converts audio to text)
|
|
- `speechToTranslatedText(audioContent: bytes, fromLanguage: str, toLanguage: str, sampleRate: Optional[int], channels: Optional[int]) → Dict[str, Any]` (converts audio to translated text)
|
|
|
|
**Text-to-Speech**:
|
|
- `textToSpeech(text: str, language: Optional[str], voice: Optional[str]) → Dict[str, Any]` (converts text to audio)
|
|
- `textToTranslatedSpeech(text: str, fromLanguage: str, toLanguage: str, voice: Optional[str]) → Dict[str, Any]` (converts text to translated audio)
|
|
|
|
**Translation**:
|
|
- `translateText(text: str, sourceLanguage: str, targetLanguage: str) → Dict[str, Any]` (translates text)
|
|
|
|
**Voice Settings** (delegates to Component interface):
|
|
- `getVoiceSettings(userId: str) → Optional[VoiceSettings]`
|
|
- `createVoiceSettings(settingsData: Dict) → Optional[VoiceSettings]`
|
|
- `updateVoiceSettings(userId: str, settingsData: Dict) → Optional[VoiceSettings]`
|
|
- `getOrCreateVoiceSettings(userId: str) → Optional[VoiceSettings]`
|
|
|
|
**Metadata**:
|
|
- `getAvailableLanguages() → Dict[str, Any]` (lists supported languages from Google API)
|
|
- `getAvailableVoices(languageCode: Optional[str]) → Dict[str, Any]` (lists available voices, optionally filtered by language)
|
|
|
|
**Operation Flow**:
|
|
|
|
```mermaid
|
|
graph TB
|
|
Request[Voice Operation Request] --> Voice[VoiceObjects]
|
|
Voice --> CheckSettings{Need Settings?}
|
|
|
|
CheckSettings -->|Yes| GetSettings[Get from Component DB]
|
|
CheckSettings -->|No| Connector[Get Connector]
|
|
|
|
GetSettings --> Connector
|
|
Connector --> GoogleAPI[Google Cloud API]
|
|
|
|
GoogleAPI --> STT{Operation Type?}
|
|
STT -->|Speech-to-Text| STTAPI[Speech-to-Text API]
|
|
STT -->|Text-to-Speech| TTSAPI[Text-to-Speech API]
|
|
STT -->|Translation| TransAPI[Translation API]
|
|
|
|
STTAPI --> Process[Process Response]
|
|
TTSAPI --> Process
|
|
TransAPI --> Process
|
|
|
|
Process --> Format[Format Response]
|
|
Format --> Return[Return Result]
|
|
```
|
|
|
|
**Supported External Systems**:
|
|
- **Google Cloud Speech-to-Text**: Audio transcription
|
|
- **Google Cloud Text-to-Speech**: Audio synthesis
|
|
- **Google Cloud Translation**: Text translation
|
|
|
|
**Key Features**:
|
|
- Multi-language support (detects and supports many languages)
|
|
- Audio format handling (auto-detects sample rate, channels)
|
|
- Combined operations (speech-to-translated-text, text-to-translated-speech)
|
|
- User voice preferences (stored in Component database)
|
|
- Language and voice discovery (queries Google API for available options)
|
|
- Error handling with detailed error messages
|
|
|
|
### Access Control Summary
|
|
|
|
**Privilege Levels**:
|
|
1. **SYSADMIN**: Full system access, all mandates
|
|
2. **ADMIN**: Full access within mandate
|
|
3. **USER**: Access to own records only
|
|
|
|
**Access Methods**:
|
|
- `uam()`: Filters recordsets by privilege, adds `_hideView`, `_hideEdit`, `_hideDelete` flags
|
|
- `canModify()`: Checks if user can create/update/delete records based on ownership and privilege
|
|
|
|
**Singleton Pattern**: Interfaces use factory functions that cache instances per user context for efficient memory usage.
|
|
|
|
**User Context**: All database interfaces require user context (User object with mandateId, userId, privilege) for access control and data filtering.
|
|
|
|
--- |