gateway/docs/SERVICE_ARCHITECTURE_DOCUMENTATION.md

# Gateway Service Architecture Documentation

This document describes the structure, design patterns, and key components of the two service architectures in the gateway:

1. **`modules/serviceCenter`** — the new service center (context-based DI, RBAC-aware)
2. **`modules/services`** — the legacy services hub (monolithic hub, eager loading)

---

## 1. `modules/serviceCenter` — New Service Center

### 1.1 File/Folder Structure

```
modules/serviceCenter/
├── __init__.py           # Public API: getService, preWarm, registerServiceObjects, can_access_service
├── context.py            # ServiceCenterContext dataclass
├── registry.py           # Service definitions (CORE_SERVICES, IMPORTABLE_SERVICES, RBAC)
├── resolver.py           # Resolution logic, dependency injection, legacy fallback
├── core/                 # Core services (internal building blocks, no RBAC)
│   ├── __init__.py
│   ├── serviceUtils/
│   │   └── mainServiceUtils.py
│   ├── serviceSecurity/
│   │   └── mainServiceSecurity.py
│   └── serviceStreaming/
│       └── mainServiceStreaming.py
└── services/             # Feature-facing importable services (RBAC-protected)
    ├── __init__.py
    ├── serviceAi/
    ├── serviceBilling/
    ├── serviceChat/
    ├── serviceExtraction/
    ├── serviceGeneration/
    ├── serviceMessaging/
    ├── serviceNeutralization/
    ├── serviceSharepoint/
    ├── serviceTicket/
    └── serviceWeb/
```

**Design distinction:**
- **Core services** — Internal utilities (utils, security, streaming). Never directly requested by features. No RBAC. Security and streaming are fully migrated; legacy hub delegates to service center.
- **Importable services** — Feature-facing. Have `objectKey` for RBAC (e.g. `service.web`, `service.extraction`).

---

### 1.2 Service Registration and Discovery

**Registration:** Services are declared statically in `registry.py`.

- **CORE_SERVICES**: Internal services with `module`, `class`, `dependencies`.
- **IMPORTABLE_SERVICES**: Feature-facing services with `module`, `class`, `dependencies`, `objectKey`, `label`.
- **SERVICE_RBAC_OBJECTS**: Derived from IMPORTABLE_SERVICES for catalog registration.

**Discovery:** No dynamic discovery. All services are explicitly listed in the registry. Adding a service requires editing `registry.py`.

```python
# Example from registry.py
IMPORTABLE_SERVICES = {
    "ai": {
        "module": "modules.serviceCenter.services.serviceAi.mainServiceAi",
        "class": "AiService",
        "dependencies": ["chat", "utils", "extraction", "billing"],
        "objectKey": "service.ai",
        "label": {"en": "AI", "de": "KI", "fr": "IA"},
    },
    # ...
}
```

---

### 1.3 Dependency Injection and Factory Patterns

**Constructor pattern:** Services receive two arguments from the resolver:
1. `context: ServiceCenterContext` — user, mandate_id, feature_instance_id, workflow
2. `get_service: Callable[[str], Any]` — function to resolve other services by key

```python
# Service Center service constructor
def __init__(self, context, get_service: Callable[[str], Any]):
    self._context = context
    self._get_service = get_service
```

**Dependency resolution:**
- The resolver (`resolver.py`) builds a `get_service` callable that recursively resolves dependencies.
- Dependencies are declared in the registry (e.g. `"dependencies": ["chat", "utils", "extraction", "billing"]`).
- Circular dependencies are detected and raise `RuntimeError`.
- Resolution is cached per `(user, mandate_id, feature_instance_id)` + `key`.

**Legacy fallback:** If a service fails to load from the service center, the resolver falls back to the legacy `Services` hub when `legacy_hub` is provided.

---

### 1.4 Main Entry Points and Usage Patterns

| Entry Point | Purpose |
|-------------|---------|
| `getService(key, context, legacy_hub=None)` | Resolve a service by key for the given context |
| `preWarm(service_keys=None)` | Pre-load service modules at startup (called in `app.py` lifespan) |
| `registerServiceObjects(catalogService)` | Register service RBAC objects (called via `registerAllFeaturesInCatalog`) |
| `can_access_service(user, rbac, service_key, ...)` | RBAC check for service access |
| `ServiceCenterContext(user, mandate_id, feature_instance_id, workflow)` | Context dataclass |

**Typical usage (chatbot feature):**
```python
from modules.serviceCenter import getService
from modules.serviceCenter.context import ServiceCenterContext

ctx = ServiceCenterContext(user=user, mandate_id=mandateId, feature_instance_id=featureInstanceId, workflow=workflow)
ai_service = getService("ai", ctx, legacy_hub=None)
chat_service = getService("chat", ctx, legacy_hub=None)
```

**Feature-level hub (e.g. chatbot):**
- `getChatbotServices()` builds a lightweight hub with only required services.
- Uses `REQUIRED_SERVICES` to resolve only `chat`, `ai`, `billing`, `streaming`.
- Returns a `_ChatbotServiceHub` object with `chat`, `ai`, `billing`, `streaming`, `interfaceDbComponent`, etc.

---

### 1.5 Initialization and Bootstrapping

1. **`app.py` lifespan:**
   - `registerAllFeaturesInCatalog(catalogService)` → calls `registerServiceObjects(catalogService)` for service RBAC objects
   - `preWarm()` — imports all service modules to avoid first-request latency

2. **`registerAllFeaturesInCatalog` (modules/system/registry.py):**
   - Registers system RBAC objects
   - Registers service center RBAC objects via `registerServiceObjects`
   - Registers feature RBAC objects

3. **First request:**
   - `getService(key, ctx)` → `resolve()` loads module, instantiates class with `(context, get_service)`, caches instance

---

## 2. `modules/services` — Legacy Services Hub

### 2.1 File/Folder Structure

```
modules/services/
├── __init__.py           # Services class, getInterface(), PublicService, _loadFeatureInterfaces, _loadFeatureServices
├── serviceAi/
│   └── mainServiceAi.py
├── serviceBilling/
│   └── mainServiceBilling.py
├── serviceChat/
│   └── mainServiceChat.py
├── serviceExtraction/
│   ├── extractors/
│   ├── chunking/
│   ├── merging/
│   ├── subRegistry.py
│   ├── subPipeline.py
│   └── mainServiceExtraction.py
├── serviceGeneration/
│   ├── paths/
│   ├── renderers/
│   └── mainServiceGeneration.py
├── serviceMessaging/
│   └── mainServiceMessaging.py
├── serviceNormalization/
│   └── mainServiceNormalization.py
├── serviceSharepoint/
│   └── mainServiceSharepoint.py
├── serviceStreaming/
│   ├── eventManager.py
│   ├── helpers.py
│   └── mainServiceStreaming.py
├── serviceTicket/
│   └── mainServiceTicket.py
├── serviceUtils/
│   └── mainServiceUtils.py
├── serviceWeb/
│   └── mainServiceWeb.py
└── serviceSecurity/
    └── mainServiceSecurity.py
```

**No core vs. services split.** All services live in flat `serviceX/` directories.

---

### 2.2 Service Registration and Discovery

**Registration:** Services are **eagerly loaded** in `Services.__init__()` via hardcoded imports. No registry file.

**Discovery:**
- **Shared services:** Loaded explicitly in `__init__` from `modules/services/serviceX/mainServiceX.py`.
- **Feature services:** Discovered dynamically via `_loadFeatureServices()` — scans `modules/features/*/service*/mainService*.py` and instantiates classes ending with `"Service"`.

```python
# Shared services — hardcoded in Services.__init__
from .serviceSharepoint.mainServiceSharepoint import SharepointService
self.sharepoint = PublicService(SharepointService(self))
from .serviceChat.mainServiceChat import ChatService
self.chat = PublicService(ChatService(self))
# ... etc.
```

---

### 2.3 Dependency Injection / Factory Patterns

**Constructor pattern:** Services receive the entire `Services` hub as their single dependency.

```python
# Legacy service constructor
def __init__(self, services):
    self.services = services
```

**No explicit dependency graph.** Services access other services via `self.services.<attr>` (e.g. `self.services.interfaceDbComponent`, `self.services.extraction`). All services are loaded at construction time.

**PublicService proxy:** Services are wrapped in `PublicService(target, functionsOnly=True)` to expose only callable, non-private attributes. Reduces accidental access to internal state.

**BillingService:** Uses a separate factory `getService(currentUser, mandateId, featureInstanceId, featureCode)` and a module-level cache. Not integrated with the hub’s constructor pattern.

---

### 2.4 Main Entry Points and Usage Patterns

| Entry Point | Purpose |
|-------------|---------|
| `getInterface(user, workflow, mandateId, featureInstanceId)` | Returns a `Services` instance |
| `Services` | Central hub with all services and interfaces |

**Typical usage:**
```python
from modules.services import getInterface as getServices

services = getServices(user, workflow, mandateId=mandateId, featureInstanceId=featureInstanceId)
ai = services.ai
extraction = services.extraction
```

**Interfaces loaded at construction:**
- `interfaceDbApp`, `interfaceDbComponent`, `interfaceDbChat`, `rbac`
- Plus dynamically loaded `interfaceFeature*` from feature containers

---

### 2.5 Initialization and Bootstrapping

1. **No startup bootstrap** — services load on first `getInterface()` call.
2. **Construction flow:**
   - `getInterface(user, ...)` → `Services(user, ...)`
   - `Services.__init__`:
     - Loads DB interfaces (`interfaceDbApp`, `interfaceDbComponent`, `interfaceDbChat`)
     - Instantiates all shared services (sharepoint, ticket, chat, utils, security, streaming, ai, extraction, generation, web)
     - Calls `_loadFeatureInterfaces()` — discovers `interfaceFeature*.py` in features
     - Calls `_loadFeatureServices()` — discovers `service*/mainService*.py` in features, overrides hub attributes

3. **Feature services:** If a feature defines `serviceAi/mainServiceAi.py`, it overrides `services.ai`. Shared `serviceAi` is only used when no feature override exists.

---

## 3. Side-by-Side Comparison

| Aspect | Service Center | Legacy Services |
|--------|----------------|-----------------|
| **Location** | `modules/serviceCenter/` | `modules/services/` |
| **Entry point** | `getService(key, context, legacy_hub)` | `getInterface(user, ...)` → `Services` |
| **Constructor** | `(context, get_service)` | `(services)` — full hub |
| **Context** | `ServiceCenterContext` (user, mandate_id, feature_instance_id, workflow) | Full `Services` with interfaces |
| **Dependencies** | Declared in registry, resolved lazily via `get_service("key")` | Via `self.services.<attr>` |
| **Loading** | Lazy (on first `getService`), cached per context | Eager (all at construction) |
| **RBAC** | Per-service `objectKey` in registry, `can_access_service()` | Shared via hub `.rbac` |
| **Feature services** | Not applicable — features use `getService(key, ctx)` | Discovered via `_loadFeatureServices()` |
| **Pre-warm** | `preWarm()` in app lifespan | None |
| **Bootstrap** | `registerServiceObjects()` via `registerAllFeaturesInCatalog` | None |

---

## 4. Coexistence and Migration

- **Service center** can fall back to **legacy hub** when `legacy_hub` is passed to `getService`.
- **Chatbot** uses service center via `getChatbotServices()` and does not use the legacy hub.
- **Billing, routes, teamsbot, commcoach, etc.** still use `modules.services` (e.g. `getInterface`, `getService` from `serviceBilling`).
- **`ServiceCenterContext`** is used when calling `getService`. Features that pass `workflow=None` use a placeholder workflow for billing/featureCode.
- Migration plan: `docs/SERVICE_CENTER_MIGRATION_PLAN.md`.

---

## 5. Service Center Resolver Flow

```
getService("ai", ctx, legacy_hub)
    → resolve("ai", ctx, cache, resolving, legacy_hub)
    → Check cache (cache_key = user_mandate_feature_ai)
    → If cache hit: return cached instance
    → If miss:
        → _load_service_class("modules.serviceCenter.services.serviceAi.mainServiceAi", "AiService")
        → Resolve dependencies: chat, utils, extraction, billing (recursive resolve)
        → instance = AiService(ctx, get_service)
        → cache[cache_key] = instance
        → return instance
    → On ImportError/ModuleNotFoundError: _get_from_legacy(legacy_hub, "ai") if legacy_hub
```

---

## 6. Key Files Reference

| File | Purpose |
|------|---------|
| `serviceCenter/registry.py` | Service definitions, dependency graph, RBAC keys |
| `serviceCenter/resolver.py` | Resolution logic, caching, legacy fallback |
| `serviceCenter/context.py` | `ServiceCenterContext` dataclass |
| `serviceCenter/__init__.py` | `getService`, `preWarm`, `registerServiceObjects`, `can_access_service` |
| `services/__init__.py` | `Services` class, `getInterface`, `PublicService`, feature discovery |
| `system/registry.py` | `registerAllFeaturesInCatalog` (calls `registerServiceObjects`) |
| `app.py` | Lifespan: `preWarm()`, `registerAllFeaturesInCatalog()` |
| `features/chatbot/mainChatbot.py` | Example: `getChatbotServices()` using service center |