209 lines
10 KiB
Markdown
209 lines
10 KiB
Markdown
# Architecture Overview
|
|
|
|
High-level architecture diagram of the Gateway project.
|
|
|
|
```mermaid
|
|
graph TB
|
|
%% Entry Point
|
|
App[app.py<br/>FastAPI Application]
|
|
|
|
%% Middleware Layer
|
|
App --> Security[Security<br/>Auth, CSRF, JWT, Token Refresh]
|
|
App --> CORS[CORS Middleware]
|
|
|
|
%% API Layer
|
|
App --> Routes[Routes<br/>API Endpoints]
|
|
|
|
%% Business Logic Layer
|
|
Routes --> Features[Features<br/>Business Logic Modules]
|
|
Routes --> Services[Services<br/>Service Layer]
|
|
|
|
%% Features can use Services
|
|
Features --> Services
|
|
|
|
%% Data Access Layer
|
|
Services --> Interfaces[Interfaces<br/>Data Access Layer]
|
|
Features --> Interfaces
|
|
|
|
%% External Connections
|
|
Interfaces --> Connectors[Connectors<br/>External System Connections]
|
|
Interfaces --> Database[(Database<br/>PostgreSQL)]
|
|
|
|
%% Connectors connect to external systems
|
|
Connectors --> Database
|
|
Connectors --> External[External Systems<br/>Jira, ClickUp, Google, etc.]
|
|
|
|
%% Shared Resources
|
|
App -.-> Shared[Shared Modules<br/>Configuration, Logging, Utils]
|
|
Routes -.-> Shared
|
|
Features -.-> Shared
|
|
Services -.-> Shared
|
|
Interfaces -.-> Shared
|
|
|
|
%% Data Models used throughout
|
|
Routes -.-> DataModels[Data Models<br/>Request/Response Schemas]
|
|
Features -.-> DataModels
|
|
Services -.-> DataModels
|
|
Interfaces -.-> DataModels
|
|
|
|
%% Feature Lifecycle Management
|
|
App --> FeaturesLifecycle[Features Lifecycle<br/>Startup/Shutdown Management]
|
|
FeaturesLifecycle --> Features
|
|
|
|
%% Styling
|
|
classDef entryPoint fill:#e1f5ff,stroke:#01579b,stroke-width:3px
|
|
classDef apiLayer fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
|
|
classDef businessLogic fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px
|
|
classDef dataAccess fill:#fff3e0,stroke:#e65100,stroke-width:2px
|
|
classDef external fill:#fce4ec,stroke:#880e4f,stroke-width:2px
|
|
classDef shared fill:#f5f5f5,stroke:#424242,stroke-width:1px,stroke-dasharray: 5 5
|
|
|
|
class App entryPoint
|
|
class Routes,Security,CORS apiLayer
|
|
class Features,Services businessLogic
|
|
class Interfaces,Connectors dataAccess
|
|
class Database,External external
|
|
class Shared,DataModels,FeaturesLifecycle shared
|
|
```
|
|
|
|
## Data Flow Diagram
|
|
|
|
The following sequence diagram shows how data flows through the same architectural layers from the architecture diagram above.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client
|
|
participant App as app.py<br/>FastAPI Application
|
|
participant Security as Security<br/>Auth, CSRF, JWT
|
|
participant Routes as Routes<br/>API Endpoints
|
|
participant Features as Features<br/>Business Logic
|
|
participant Services as Services<br/>Service Layer
|
|
participant Interfaces as Interfaces<br/>Data Access Layer
|
|
participant Connectors as Connectors<br/>External Connections
|
|
participant Database as Database<br/>PostgreSQL
|
|
participant External as External Systems<br/>Jira, ClickUp, etc.
|
|
|
|
%% Request Flow
|
|
Client->>App: HTTP Request
|
|
App->>Security: Validate Auth & CSRF
|
|
Security-->>App: Authenticated
|
|
App->>Routes: Validated Request
|
|
Routes->>Routes: Validate Data Models
|
|
|
|
alt Route delegates to Features
|
|
Routes->>Features: Delegate Request
|
|
Features->>Services: Use Service (optional)
|
|
Services-->>Features: Service Result
|
|
Features->>Interfaces: Request Data Access
|
|
else Route delegates to Services
|
|
Routes->>Services: Delegate Request
|
|
Services->>Interfaces: Request Data Access
|
|
end
|
|
|
|
Interfaces->>Connectors: Query Request
|
|
|
|
alt Database Query
|
|
Connectors->>Database: Execute Query
|
|
Database-->>Connectors: Raw Data
|
|
else External API Call
|
|
Connectors->>External: API Call
|
|
External-->>Connectors: API Response
|
|
end
|
|
|
|
%% Response Flow
|
|
Connectors-->>Interfaces: Raw Data
|
|
Interfaces->>Interfaces: Transform to Domain Objects
|
|
Interfaces-->>Features: Domain Objects
|
|
Interfaces-->>Services: Domain Objects
|
|
|
|
Features->>Features: Process Business Logic
|
|
Services->>Services: Process Service Logic
|
|
|
|
Features-->>Routes: Processed Data
|
|
Services-->>Routes: Processed Data
|
|
|
|
Routes->>Routes: Serialize to Response Models
|
|
Routes-->>App: HTTP Response
|
|
App-->>Client: HTTP Response
|
|
```
|
|
|
|
### Request Flow (Top to Bottom)
|
|
|
|
1. **Client** sends HTTP request to **app.py**
|
|
2. **app.py** forwards to **Security** middleware for authentication and CSRF validation
|
|
3. **Security** validates and returns authenticated request to **app.py**
|
|
4. **app.py** forwards validated request to **Routes**
|
|
5. **Routes** validate request data using Data Models, then delegate to either:
|
|
- **Features** (which may use **Services**), or
|
|
- **Services** directly
|
|
6. **Features/Services** call **Interfaces** for data access
|
|
7. **Interfaces** use **Connectors** to execute queries
|
|
8. **Connectors** query **Database** or call **External Systems**
|
|
|
|
### Response Flow (Bottom to Top)
|
|
|
|
1. **Database/External Systems** return raw data to **Connectors**
|
|
2. **Connectors** pass raw data to **Interfaces**
|
|
3. **Interfaces** transform raw data into domain objects
|
|
4. **Interfaces** return domain objects to **Features/Services**
|
|
5. **Features/Services** process business logic and return processed data to **Routes**
|
|
6. **Routes** serialize data to response models and return HTTP response to **app.py**
|
|
7. **app.py** returns HTTP response to **Client**
|
|
|
|
### Data Transformations
|
|
|
|
- **Routes**: HTTP Request ↔ Validated Data Models (Pydantic)
|
|
- **Features/Services**: Data Models ↔ Domain Objects (Business Logic Processing)
|
|
- **Interfaces**: Domain Objects ↔ Raw Data (SQL/API Format)
|
|
- **Connectors**: Raw Data ↔ Database Queries/API Calls
|
|
|
|
## Layer Descriptions
|
|
|
|
### Entry Point Layer
|
|
**app.py** - The FastAPI application entry point that orchestrates the entire system. It initializes logging, configures CORS and security middleware, registers all route routers, and manages the application lifecycle (startup/shutdown). This is where the application server starts and all components are wired together.
|
|
|
|
### API Layer
|
|
**Routes** - HTTP endpoints that define the REST API surface. Routes receive client requests, validate input using data models, delegate to features or services for business logic, and return structured responses. Each route module handles a specific domain (e.g., Real Estate, Chat, Workflows, Security).
|
|
|
|
**Security** - Middleware and services that handle authentication, authorization, CSRF protection, JWT token management, and token refresh. Ensures all requests are properly authenticated and authorized before reaching business logic. See [Security Component Documentation](./security-component.md) for detailed documentation.
|
|
|
|
**CORS** - Cross-Origin Resource Sharing middleware that controls which external domains can access the API, enabling secure cross-origin requests from web applications.
|
|
|
|
### Business Logic Layer
|
|
**Features** - Domain-specific business logic modules that implement core functionality for specific use cases (e.g., Real Estate management, Chat workflows, Data neutralization). Features are stateless and orchestrate services to fulfill business requirements. They can be called directly from routes or managed by the Features Lifecycle for background processing.
|
|
|
|
**Services** - Reusable, composable service components that provide cross-cutting functionality (AI processing, document extraction, content generation, chat operations, ticket management, etc.). Services encapsulate complex operations and can be used by multiple features. They typically use interfaces to access data and may call other services.
|
|
|
|
### Data Access Layer
|
|
**Interfaces** - Abstraction layer that provides a clean, domain-oriented API for accessing data. Interfaces hide the complexity of database connections and external system integrations, offering high-level methods for CRUD operations. They handle user context, access control, and data transformation between the application and persistence layers.
|
|
|
|
**Connectors** - Concrete implementations that handle low-level communication with external systems. Database connectors manage PostgreSQL connections, query execution, and transaction handling. External connectors integrate with third-party services (Jira, ClickUp, Google Voice, SharePoint) using their specific APIs and protocols.
|
|
|
|
### External Systems Layer
|
|
**Database** - PostgreSQL databases that persist application data. Multiple databases may exist for different domains (e.g., chat data, real estate data, management data). Connectors handle all database interactions.
|
|
|
|
**External Systems** - Third-party services and APIs that the application integrates with. These include ticketing systems (Jira, ClickUp), cloud services (Google Voice, SharePoint), and other external platforms. Connectors abstract away the specifics of each integration.
|
|
|
|
### Shared Resources Layer
|
|
**Shared Modules** - Common utilities and infrastructure used throughout the application. Includes configuration management, logging utilities, time/date helpers, JSON processing, attribute utilities, and audit logging. These modules provide cross-cutting concerns that don't belong to any specific domain.
|
|
|
|
**Data Models** - Pydantic models that define data structures for requests, responses, and database entities. They provide validation, serialization, and type safety across all layers. Models are organized by domain (e.g., Real Estate, Chat, Security, AI).
|
|
|
|
**Features Lifecycle** - Manages the startup and shutdown of features that require background processing, scheduled tasks, or event-driven operations. Coordinates initialization and cleanup of features that need persistent processes or event listeners.
|
|
|
|
## Request/Response Flow Summary
|
|
|
|
For a detailed visual representation, see the [Data Flow Diagram](#data-flow-diagram) above.
|
|
|
|
**Simplified Request Flow**: `Client Request` → `CORS` → `Security (Auth/CSRF)` → `Routes` → `Features/Services` → `Interfaces` → `Connectors` → `Database/External Systems`
|
|
|
|
**Simplified Response Flow**: `Database/External Systems` → `Connectors` → `Interfaces` → `Features/Services` → `Routes` → `Transform & Log` → `Client Response`
|
|
|
|
## Key Architectural Patterns
|
|
|
|
- **Layered Architecture**: Clear separation between API, business logic, and data access layers
|
|
- **Dependency Injection**: Services and interfaces are injected where needed
|
|
- **Interface Abstraction**: Interfaces abstract away database and connector details
|
|
- **Stateless Design**: Features operate statelessly without session management
|
|
- **Shared Utilities**: Common functionality centralized in shared modules
|
|
|