# Architecture Overview

High-level architecture diagram of the Gateway project.

```mermaid
graph TB
    %% Entry Point
    App[app.py<br/>FastAPI Application]
    
    %% Middleware Layer
    App --> Security[Security<br/>Auth, CSRF, JWT, Token Refresh]
    App --> CORS[CORS Middleware]
    
    %% API Layer
    App --> Routes[Routes<br/>API Endpoints]
    
    %% Business Logic Layer
    Routes --> Features[Features<br/>Business Logic Modules]
    Routes --> Services[Services<br/>Service Layer]
    
    %% Features can use Services
    Features --> Services
    
    %% Data Access Layer
    Services --> Interfaces[Interfaces<br/>Data Access Layer]
    Features --> Interfaces
    
    %% External Connections
    Interfaces --> Connectors[Connectors<br/>External System Connections]
    Interfaces --> Database[(Database<br/>PostgreSQL)]
    
    %% Connectors connect to external systems
    Connectors --> Database
    Connectors --> External[External Systems<br/>Jira, ClickUp, Google, etc.]
    
    %% Shared Resources
    App -.-> Shared[Shared Modules<br/>Configuration, Logging, Utils]
    Routes -.-> Shared
    Features -.-> Shared
    Services -.-> Shared
    Interfaces -.-> Shared
    
    %% Data Models used throughout
    Routes -.-> DataModels[Data Models<br/>Request/Response Schemas]
    Features -.-> DataModels
    Services -.-> DataModels
    Interfaces -.-> DataModels
    
    %% Feature Lifecycle Management
    App --> FeaturesLifecycle[Features Lifecycle<br/>Startup/Shutdown Management]
    FeaturesLifecycle --> Features
    
    %% Styling
    classDef entryPoint fill:#e1f5ff,stroke:#01579b,stroke-width:3px
    classDef apiLayer fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef businessLogic fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px
    classDef dataAccess fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef external fill:#fce4ec,stroke:#880e4f,stroke-width:2px
    classDef shared fill:#f5f5f5,stroke:#424242,stroke-width:1px,stroke-dasharray: 5 5
    
    class App entryPoint
    class Routes,Security,CORS apiLayer
    class Features,Services businessLogic
    class Interfaces,Connectors dataAccess
    class Database,External external
    class Shared,DataModels,FeaturesLifecycle shared
```

## Data Flow Diagram

The following sequence diagram shows how data flows through the same architectural layers from the architecture diagram above.

```mermaid
sequenceDiagram
    participant Client
    participant App as app.py<br/>FastAPI Application
    participant Security as Security<br/>Auth, CSRF, JWT
    participant Routes as Routes<br/>API Endpoints
    participant Features as Features<br/>Business Logic
    participant Services as Services<br/>Service Layer
    participant Interfaces as Interfaces<br/>Data Access Layer
    participant Connectors as Connectors<br/>External Connections
    participant Database as Database<br/>PostgreSQL
    participant External as External Systems<br/>Jira, ClickUp, etc.
    
    %% Request Flow
    Client->>App: HTTP Request
    App->>Security: Validate Auth & CSRF
    Security-->>App: Authenticated
    App->>Routes: Validated Request
    Routes->>Routes: Validate Data Models
    
    alt Route delegates to Features
        Routes->>Features: Delegate Request
        Features->>Services: Use Service (optional)
        Services-->>Features: Service Result
        Features->>Interfaces: Request Data Access
    else Route delegates to Services
        Routes->>Services: Delegate Request
        Services->>Interfaces: Request Data Access
    end
    
    Interfaces->>Connectors: Query Request
    
    alt Database Query
        Connectors->>Database: Execute Query
        Database-->>Connectors: Raw Data
    else External API Call
        Connectors->>External: API Call
        External-->>Connectors: API Response
    end
    
    %% Response Flow
    Connectors-->>Interfaces: Raw Data
    Interfaces->>Interfaces: Transform to Domain Objects
    Interfaces-->>Features: Domain Objects
    Interfaces-->>Services: Domain Objects
    
    Features->>Features: Process Business Logic
    Services->>Services: Process Service Logic
    
    Features-->>Routes: Processed Data
    Services-->>Routes: Processed Data
    
    Routes->>Routes: Serialize to Response Models
    Routes-->>App: HTTP Response
    App-->>Client: HTTP Response
```

### Request Flow (Top to Bottom)

1. **Client** sends HTTP request to **app.py**
2. **app.py** forwards to **Security** middleware for authentication and CSRF validation
3. **Security** validates and returns authenticated request to **app.py**
4. **app.py** forwards validated request to **Routes**
5. **Routes** validate request data using Data Models, then delegate to either:
   - **Features** (which may use **Services**), or
   - **Services** directly
6. **Features/Services** call **Interfaces** for data access
7. **Interfaces** use **Connectors** to execute queries
8. **Connectors** query **Database** or call **External Systems**

### Response Flow (Bottom to Top)

1. **Database/External Systems** return raw data to **Connectors**
2. **Connectors** pass raw data to **Interfaces**
3. **Interfaces** transform raw data into domain objects
4. **Interfaces** return domain objects to **Features/Services**
5. **Features/Services** process business logic and return processed data to **Routes**
6. **Routes** serialize data to response models and return HTTP response to **app.py**
7. **app.py** returns HTTP response to **Client**

### Data Transformations

- **Routes**: HTTP Request ↔ Validated Data Models (Pydantic)
- **Features/Services**: Data Models ↔ Domain Objects (Business Logic Processing)
- **Interfaces**: Domain Objects ↔ Raw Data (SQL/API Format)
- **Connectors**: Raw Data ↔ Database Queries/API Calls

## Layer Descriptions

### Entry Point Layer
**app.py** - The FastAPI application entry point that orchestrates the entire system. It initializes logging, configures CORS and security middleware, registers all route routers, and manages the application lifecycle (startup/shutdown). This is where the application server starts and all components are wired together.

### API Layer
**Routes** - HTTP endpoints that define the REST API surface. Routes receive client requests, validate input using data models, delegate to features or services for business logic, and return structured responses. Each route module handles a specific domain (e.g., Real Estate, Chat, Workflows, Security).

**Security** - Middleware and services that handle authentication, authorization, CSRF protection, JWT token management, and token refresh. Ensures all requests are properly authenticated and authorized before reaching business logic. See [Security Component Documentation](./security-component.md) for detailed documentation.

**CORS** - Cross-Origin Resource Sharing middleware that controls which external domains can access the API, enabling secure cross-origin requests from web applications.

### Business Logic Layer
**Features** - Domain-specific business logic modules that implement core functionality for specific use cases (e.g., Real Estate management, Chat workflows, Data neutralization). Features are stateless and orchestrate services to fulfill business requirements. They can be called directly from routes or managed by the Features Lifecycle for background processing.

**Services** - Reusable, composable service components that provide cross-cutting functionality (AI processing, document extraction, content generation, chat operations, ticket management, etc.). Services encapsulate complex operations and can be used by multiple features. They typically use interfaces to access data and may call other services.

### Data Access Layer
**Interfaces** - Abstraction layer that provides a clean, domain-oriented API for accessing data. Interfaces hide the complexity of database connections and external system integrations, offering high-level methods for CRUD operations. They handle user context, access control, and data transformation between the application and persistence layers.

**Connectors** - Concrete implementations that handle low-level communication with external systems. Database connectors manage PostgreSQL connections, query execution, and transaction handling. External connectors integrate with third-party services (Jira, ClickUp, Google Voice, SharePoint) using their specific APIs and protocols.

### External Systems Layer
**Database** - PostgreSQL databases that persist application data. Multiple databases may exist for different domains (e.g., chat data, real estate data, management data). Connectors handle all database interactions.

**External Systems** - Third-party services and APIs that the application integrates with. These include ticketing systems (Jira, ClickUp), cloud services (Google Voice, SharePoint), and other external platforms. Connectors abstract away the specifics of each integration.

### Shared Resources Layer
**Shared Modules** - Common utilities and infrastructure used throughout the application. Includes configuration management, logging utilities, time/date helpers, JSON processing, attribute utilities, and audit logging. These modules provide cross-cutting concerns that don't belong to any specific domain.

**Data Models** - Pydantic models that define data structures for requests, responses, and database entities. They provide validation, serialization, and type safety across all layers. Models are organized by domain (e.g., Real Estate, Chat, Security, AI).

**Features Lifecycle** - Manages the startup and shutdown of features that require background processing, scheduled tasks, or event-driven operations. Coordinates initialization and cleanup of features that need persistent processes or event listeners.

## Request/Response Flow Summary

For a detailed visual representation, see the [Data Flow Diagram](#data-flow-diagram) above.

**Simplified Request Flow**: `Client Request` → `CORS` → `Security (Auth/CSRF)` → `Routes` → `Features/Services` → `Interfaces` → `Connectors` → `Database/External Systems`

**Simplified Response Flow**: `Database/External Systems` → `Connectors` → `Interfaces` → `Features/Services` → `Routes` → `Transform & Log` → `Client Response`

## Key Architectural Patterns

- **Layered Architecture**: Clear separation between API, business logic, and data access layers
- **Dependency Injection**: Services and interfaces are injected where needed
- **Interface Abstraction**: Interfaces abstract away database and connector details
- **Stateless Design**: Features operate statelessly without session management
- **Shared Utilities**: Common functionality centralized in shared modules