diff --git a/docs/SERVICE_ARCHITECTURE_DOCUMENTATION.md b/docs/SERVICE_ARCHITECTURE_DOCUMENTATION.md deleted file mode 100644 index b8cdd891..00000000 --- a/docs/SERVICE_ARCHITECTURE_DOCUMENTATION.md +++ /dev/null @@ -1,318 +0,0 @@ -# Gateway Service Architecture Documentation - -This document describes the structure, design patterns, and key components of the two service architectures in the gateway: - -1. **`modules/serviceCenter`** — the new service center (context-based DI, RBAC-aware) -2. **`modules/services`** — the legacy services hub (monolithic hub, eager loading) - ---- - -## 1. `modules/serviceCenter` — New Service Center - -### 1.1 File/Folder Structure - -``` -modules/serviceCenter/ -├── __init__.py # Public API: getService, preWarm, registerServiceObjects, can_access_service -├── context.py # ServiceCenterContext dataclass -├── registry.py # Service definitions (CORE_SERVICES, IMPORTABLE_SERVICES, RBAC) -├── resolver.py # Resolution logic, dependency injection, legacy fallback -├── core/ # Core services (internal building blocks, no RBAC) -│ ├── __init__.py -│ ├── serviceUtils/ -│ │ └── mainServiceUtils.py -│ ├── serviceSecurity/ -│ │ └── mainServiceSecurity.py -│ └── serviceStreaming/ -│ └── mainServiceStreaming.py -└── services/ # Feature-facing importable services (RBAC-protected) - ├── __init__.py - ├── serviceAi/ - ├── serviceBilling/ - ├── serviceChat/ - ├── serviceExtraction/ - ├── serviceGeneration/ - ├── serviceMessaging/ - ├── serviceNeutralization/ - ├── serviceSharepoint/ - ├── serviceTicket/ - └── serviceWeb/ -``` - -**Design distinction:** -- **Core services** — Internal utilities (utils, security, streaming). Never directly requested by features. No RBAC. Security and streaming are fully migrated; legacy hub delegates to service center. -- **Importable services** — Feature-facing. Have `objectKey` for RBAC (e.g. `service.web`, `service.extraction`). - ---- - -### 1.2 Service Registration and Discovery - -**Registration:** Services are declared statically in `registry.py`. - -- **CORE_SERVICES**: Internal services with `module`, `class`, `dependencies`. -- **IMPORTABLE_SERVICES**: Feature-facing services with `module`, `class`, `dependencies`, `objectKey`, `label`. -- **SERVICE_RBAC_OBJECTS**: Derived from IMPORTABLE_SERVICES for catalog registration. - -**Discovery:** No dynamic discovery. All services are explicitly listed in the registry. Adding a service requires editing `registry.py`. - -```python -# Example from registry.py -IMPORTABLE_SERVICES = { - "ai": { - "module": "modules.serviceCenter.services.serviceAi.mainServiceAi", - "class": "AiService", - "dependencies": ["chat", "utils", "extraction", "billing"], - "objectKey": "service.ai", - "label": {"en": "AI", "de": "KI", "fr": "IA"}, - }, - # ... -} -``` - ---- - -### 1.3 Dependency Injection and Factory Patterns - -**Constructor pattern:** Services receive two arguments from the resolver: -1. `context: ServiceCenterContext` — user, mandate_id, feature_instance_id, workflow -2. `get_service: Callable[[str], Any]` — function to resolve other services by key - -```python -# Service Center service constructor -def __init__(self, context, get_service: Callable[[str], Any]): - self._context = context - self._get_service = get_service -``` - -**Dependency resolution:** -- The resolver (`resolver.py`) builds a `get_service` callable that recursively resolves dependencies. -- Dependencies are declared in the registry (e.g. `"dependencies": ["chat", "utils", "extraction", "billing"]`). -- Circular dependencies are detected and raise `RuntimeError`. -- Resolution is cached per `(user, mandate_id, feature_instance_id)` + `key`. - -**Legacy fallback:** If a service fails to load from the service center, the resolver falls back to the legacy `Services` hub when `legacy_hub` is provided. - ---- - -### 1.4 Main Entry Points and Usage Patterns - -| Entry Point | Purpose | -|-------------|---------| -| `getService(key, context, legacy_hub=None)` | Resolve a service by key for the given context | -| `preWarm(service_keys=None)` | Pre-load service modules at startup (called in `app.py` lifespan) | -| `registerServiceObjects(catalogService)` | Register service RBAC objects (called via `registerAllFeaturesInCatalog`) | -| `can_access_service(user, rbac, service_key, ...)` | RBAC check for service access | -| `ServiceCenterContext(user, mandate_id, feature_instance_id, workflow)` | Context dataclass | - -**Typical usage (chatbot feature):** -```python -from modules.serviceCenter import getService -from modules.serviceCenter.context import ServiceCenterContext - -ctx = ServiceCenterContext(user=user, mandate_id=mandateId, feature_instance_id=featureInstanceId, workflow=workflow) -ai_service = getService("ai", ctx, legacy_hub=None) -chat_service = getService("chat", ctx, legacy_hub=None) -``` - -**Feature-level hub (e.g. chatbot):** -- `getChatbotServices()` builds a lightweight hub with only required services. -- Uses `REQUIRED_SERVICES` to resolve only `chat`, `ai`, `billing`, `streaming`. -- Returns a `_ChatbotServiceHub` object with `chat`, `ai`, `billing`, `streaming`, `interfaceDbComponent`, etc. - ---- - -### 1.5 Initialization and Bootstrapping - -1. **`app.py` lifespan:** - - `registerAllFeaturesInCatalog(catalogService)` → calls `registerServiceObjects(catalogService)` for service RBAC objects - - `preWarm()` — imports all service modules to avoid first-request latency - -2. **`registerAllFeaturesInCatalog` (modules/system/registry.py):** - - Registers system RBAC objects - - Registers service center RBAC objects via `registerServiceObjects` - - Registers feature RBAC objects - -3. **First request:** - - `getService(key, ctx)` → `resolve()` loads module, instantiates class with `(context, get_service)`, caches instance - ---- - -## 2. `modules/services` — Legacy Services Hub - -### 2.1 File/Folder Structure - -``` -modules/services/ -├── __init__.py # Services class, getInterface(), PublicService, _loadFeatureInterfaces, _loadFeatureServices -├── serviceAi/ -│ └── mainServiceAi.py -├── serviceBilling/ -│ └── mainServiceBilling.py -├── serviceChat/ -│ └── mainServiceChat.py -├── serviceExtraction/ -│ ├── extractors/ -│ ├── chunking/ -│ ├── merging/ -│ ├── subRegistry.py -│ ├── subPipeline.py -│ └── mainServiceExtraction.py -├── serviceGeneration/ -│ ├── paths/ -│ ├── renderers/ -│ └── mainServiceGeneration.py -├── serviceMessaging/ -│ └── mainServiceMessaging.py -├── serviceNormalization/ -│ └── mainServiceNormalization.py -├── serviceSharepoint/ -│ └── mainServiceSharepoint.py -├── serviceStreaming/ -│ ├── eventManager.py -│ ├── helpers.py -│ └── mainServiceStreaming.py -├── serviceTicket/ -│ └── mainServiceTicket.py -├── serviceUtils/ -│ └── mainServiceUtils.py -├── serviceWeb/ -│ └── mainServiceWeb.py -└── serviceSecurity/ - └── mainServiceSecurity.py -``` - -**No core vs. services split.** All services live in flat `serviceX/` directories. - ---- - -### 2.2 Service Registration and Discovery - -**Registration:** Services are **eagerly loaded** in `Services.__init__()` via hardcoded imports. No registry file. - -**Discovery:** -- **Shared services:** Loaded explicitly in `__init__` from `modules/services/serviceX/mainServiceX.py`. -- **Feature services:** Discovered dynamically via `_loadFeatureServices()` — scans `modules/features/*/service*/mainService*.py` and instantiates classes ending with `"Service"`. - -```python -# Shared services — hardcoded in Services.__init__ -from .serviceSharepoint.mainServiceSharepoint import SharepointService -self.sharepoint = PublicService(SharepointService(self)) -from .serviceChat.mainServiceChat import ChatService -self.chat = PublicService(ChatService(self)) -# ... etc. -``` - ---- - -### 2.3 Dependency Injection / Factory Patterns - -**Constructor pattern:** Services receive the entire `Services` hub as their single dependency. - -```python -# Legacy service constructor -def __init__(self, services): - self.services = services -``` - -**No explicit dependency graph.** Services access other services via `self.services.` (e.g. `self.services.interfaceDbComponent`, `self.services.extraction`). All services are loaded at construction time. - -**PublicService proxy:** Services are wrapped in `PublicService(target, functionsOnly=True)` to expose only callable, non-private attributes. Reduces accidental access to internal state. - -**BillingService:** Uses a separate factory `getService(currentUser, mandateId, featureInstanceId, featureCode)` and a module-level cache. Not integrated with the hub’s constructor pattern. - ---- - -### 2.4 Main Entry Points and Usage Patterns - -| Entry Point | Purpose | -|-------------|---------| -| `getInterface(user, workflow, mandateId, featureInstanceId)` | Returns a `Services` instance | -| `Services` | Central hub with all services and interfaces | - -**Typical usage:** -```python -from modules.services import getInterface as getServices - -services = getServices(user, workflow, mandateId=mandateId, featureInstanceId=featureInstanceId) -ai = services.ai -extraction = services.extraction -``` - -**Interfaces loaded at construction:** -- `interfaceDbApp`, `interfaceDbComponent`, `interfaceDbChat`, `rbac` -- Plus dynamically loaded `interfaceFeature*` from feature containers - ---- - -### 2.5 Initialization and Bootstrapping - -1. **No startup bootstrap** — services load on first `getInterface()` call. -2. **Construction flow:** - - `getInterface(user, ...)` → `Services(user, ...)` - - `Services.__init__`: - - Loads DB interfaces (`interfaceDbApp`, `interfaceDbComponent`, `interfaceDbChat`) - - Instantiates all shared services (sharepoint, ticket, chat, utils, security, streaming, ai, extraction, generation, web) - - Calls `_loadFeatureInterfaces()` — discovers `interfaceFeature*.py` in features - - Calls `_loadFeatureServices()` — discovers `service*/mainService*.py` in features, overrides hub attributes - -3. **Feature services:** If a feature defines `serviceAi/mainServiceAi.py`, it overrides `services.ai`. Shared `serviceAi` is only used when no feature override exists. - ---- - -## 3. Side-by-Side Comparison - -| Aspect | Service Center | Legacy Services | -|--------|----------------|-----------------| -| **Location** | `modules/serviceCenter/` | `modules/services/` | -| **Entry point** | `getService(key, context, legacy_hub)` | `getInterface(user, ...)` → `Services` | -| **Constructor** | `(context, get_service)` | `(services)` — full hub | -| **Context** | `ServiceCenterContext` (user, mandate_id, feature_instance_id, workflow) | Full `Services` with interfaces | -| **Dependencies** | Declared in registry, resolved lazily via `get_service("key")` | Via `self.services.` | -| **Loading** | Lazy (on first `getService`), cached per context | Eager (all at construction) | -| **RBAC** | Per-service `objectKey` in registry, `can_access_service()` | Shared via hub `.rbac` | -| **Feature services** | Not applicable — features use `getService(key, ctx)` | Discovered via `_loadFeatureServices()` | -| **Pre-warm** | `preWarm()` in app lifespan | None | -| **Bootstrap** | `registerServiceObjects()` via `registerAllFeaturesInCatalog` | None | - ---- - -## 4. Coexistence and Migration - -- **Service center** can fall back to **legacy hub** when `legacy_hub` is passed to `getService`. -- **Chatbot** uses service center via `getChatbotServices()` and does not use the legacy hub. -- **Billing, routes, teamsbot, commcoach, etc.** still use `modules.services` (e.g. `getInterface`, `getService` from `serviceBilling`). -- **`ServiceCenterContext`** is used when calling `getService`. Features that pass `workflow=None` use a placeholder workflow for billing/featureCode. -- Migration plan: `docs/SERVICE_CENTER_MIGRATION_PLAN.md`. - ---- - -## 5. Service Center Resolver Flow - -``` -getService("ai", ctx, legacy_hub) - → resolve("ai", ctx, cache, resolving, legacy_hub) - → Check cache (cache_key = user_mandate_feature_ai) - → If cache hit: return cached instance - → If miss: - → _load_service_class("modules.serviceCenter.services.serviceAi.mainServiceAi", "AiService") - → Resolve dependencies: chat, utils, extraction, billing (recursive resolve) - → instance = AiService(ctx, get_service) - → cache[cache_key] = instance - → return instance - → On ImportError/ModuleNotFoundError: _get_from_legacy(legacy_hub, "ai") if legacy_hub -``` - ---- - -## 6. Key Files Reference - -| File | Purpose | -|------|---------| -| `serviceCenter/registry.py` | Service definitions, dependency graph, RBAC keys | -| `serviceCenter/resolver.py` | Resolution logic, caching, legacy fallback | -| `serviceCenter/context.py` | `ServiceCenterContext` dataclass | -| `serviceCenter/__init__.py` | `getService`, `preWarm`, `registerServiceObjects`, `can_access_service` | -| `services/__init__.py` | `Services` class, `getInterface`, `PublicService`, feature discovery | -| `system/registry.py` | `registerAllFeaturesInCatalog` (calls `registerServiceObjects`) | -| `app.py` | Lifespan: `preWarm()`, `registerAllFeaturesInCatalog()` | -| `features/chatbot/mainChatbot.py` | Example: `getChatbotServices()` using service center | diff --git a/docs/SERVICE_CENTER_MIGRATION_PLAN.md b/docs/SERVICE_CENTER_MIGRATION_PLAN.md deleted file mode 100644 index 450965ea..00000000 --- a/docs/SERVICE_CENTER_MIGRATION_PLAN.md +++ /dev/null @@ -1,217 +0,0 @@ -# Service Center Migration Plan - -## Overview - -This document describes a **step-by-step plan** to migrate from the old `modules/services` (Services hub) to the new `modules/serviceCenter`. The migration is **incremental**—one feature at a time—with UI-driven testing after each step. - -**Recommended first feature: Chatbot** — it has a clear UI, limited service dependencies, and is already partially using the service center (AI, generation, billing). - ---- - -## Architecture Summary - -### Current State - -| Component | Location | Notes | -|-----------|----------|-------| -| **Service Center** | `modules/serviceCenter/` | New: registry, resolver, context-based DI | -| **Services Hub** | `modules/services/` | Legacy: `getInterface()` → `Services` instance | -| **Chatbot** | `modules/features/chatbot/` | Uses `getServices()` → `.chat`, `.ai` | - -### Service Center vs Legacy Services - -| Aspect | Service Center | Legacy Services | -|--------|----------------|-----------------| -| **Constructor** | `(context: ServiceCenterContext, get_service)` | `(services: Services)` — receives hub | -| **Context** | Minimal: user, mandate_id, feature_instance_id, workflow | Full hub with all interfaces | -| **Dependencies** | Injected via `get_service("key")` | Via `self.services.` | -| **RBAC** | Per-service `objectKey` in registry | Shared via hub | -| **Pre-warm** | `preWarm()` at app startup | Loaded on first use | - -### Services Already Using Service Center (in Services class) - -The `Services` class in `modules/services/__init__.py` already uses `getService()` for: - -- `messaging` -- `ai` -- `generation` -- `billing` - -### Services Still Using Legacy Direct Imports - -- `chat` ← **Target for Phase 1** -- `sharepoint` -- `ticket` -- `utils` -- `security` -- `streaming` -- `extraction` -- `web` - ---- - -## Phase 1: Migrate Chatbot to Use Service Center for Chat - -**Goal:** Switch the Chatbot feature to get the Chat service from Service Center instead of the legacy hub. This validates the full flow with minimal risk. - -### Step 1.1: Switch Services Class to Use Service Center for Chat - -**File:** `modules/services/__init__.py` - -**Change:** Replace the direct ChatService import with `getService("chat", ...)`. - -```python -# BEFORE (line ~126-127): -from .serviceChat.mainServiceChat import ChatService -self.chat = PublicService(ChatService(self)) - -# AFTER: -self.chat = PublicService(getService("chat", _ctx, legacy_hub=self)) -``` - -The `_ctx` (ServiceCenterContext) is already created for messaging/ai/generation. Add `workflow=self.workflow` to the context if not already present (it should be—check the existing `_ctx` creation around line 109–116). - -**Verification:** -1. Ensure `ServiceCenterContext` includes `workflow` when Services has one (chatbot often passes `workflow=None` initially). -2. The service center ChatService gets `interfaceDbComponent` from `getInterface(context.user, mandateId=context.mandate_id)` — same as legacy. The chatbot calls `getFileInfo(fileId)` which only needs `interfaceDbComponent`, not workflow. - -### Step 1.2: Ensure Service Center Context Has Workflow - -**File:** `modules/services/__init__.py` - -Verify the existing context creation: - -```python -_ctx = ServiceCenterContext( - user=self.user, - mandate_id=self.mandateId, - feature_instance_id=self.featureInstanceId, - workflow=self.workflow, -) -``` - -If `workflow` is missing, add it. The ChatService uses `_context.workflow` for methods like `getChatDocumentsFromDocumentList`; for `getFileInfo` it is not needed. - -### Step 1.3: Run Unit Tests - -```powershell -cd c:\Users\IdaDittrich\Documents\01_Code\gateway -pytest tests/unit/serviceCenter/test_service_center_imports.py -v -python tests/scripts/smoke_test_service_center.py -``` - -### Step 1.4: Manual UI Test — Chatbot with File Upload - -1. **Start the gateway:** - ```powershell - cd c:\Users\IdaDittrich\Documents\01_Code\gateway - uvicorn app:app --reload --host 0.0.0.0 --port 8000 - ``` - -2. **Start the frontend** (if using frontend_nyla): - ```powershell - cd c:\Users\IdaDittrich\Documents\01_Code\frontend_nyla - npm run dev - ``` - -3. **Log in** as a user with access to the Chatbot feature. - -4. **Open a Chatbot instance** (navigate to the chatbot feature, select or create an instance). - -5. **Create a new conversation** — click "New conversation" or equivalent. - -6. **Attach a file** — upload a PDF or document before sending. - -7. **Send a message** — e.g. "Summarize this document." - -8. **Verify:** - - No 500 errors in gateway logs - - File is processed (chat service’s `getFileInfo` is used when creating `ChatbotDocument`s) - - AI response streams back correctly (AI service already from service center) - -### Step 1.5: Rollback if Needed - -If something breaks, revert the change in `modules/services/__init__.py`: - -```python -from .serviceChat.mainServiceChat import ChatService -self.chat = PublicService(ChatService(self)) -``` - ---- - -## Phase 2 (Future): Migrate Extraction for Chatbot - -The chatbot may use extraction when processing documents. After Phase 1 is stable: - -1. Switch `Services` to use `getService("extraction", _ctx, legacy_hub=self)` instead of direct import. -2. Ensure `ExtractionService` in service center has the same interface as the legacy one. -3. Re-test chatbot with document-heavy prompts. - ---- - -## Phase 3 (Future): Migrate Remaining Services - -| Service | Used By | Priority | -|---------|---------|----------| -| utils | Chat, Extraction, AI, Web, Generation | High (core) | -| security | Sharepoint | Medium | -| streaming | Workflows, Chatbot SSE | Medium | -| sharepoint | Sharepoint workflows | Medium | -| ticket | Ticket system | Low | -| web | Web research workflows | Medium | - ---- - -## Service Center Bootstrap (Already Done) - -The app already: -- Calls `preWarm()` at startup (`app.py` lifespan) -- Has `registerServiceObjects()` available for RBAC catalog (call from bootstrap if needed) - -### Optional: Register Service RBAC Objects - -If you want service-level RBAC (e.g. `can_access_service()`), call during bootstrap: - -```python -# In app.py lifespan or interfaceBootstrap -from modules.serviceCenter import registerServiceObjects -from modules.security.rbacCatalog import getCatalogService -catalogService = getCatalogService() -registerServiceObjects(catalogService) -``` - ---- - -## Testing Checklist (Chatbot Phase 1) - -- [ ] Unit tests pass: `pytest tests/unit/serviceCenter/ -v` -- [ ] Smoke test passes: `python tests/scripts/smoke_test_service_center.py` -- [ ] Gateway starts without import errors -- [ ] Chatbot UI loads -- [ ] New conversation creates successfully -- [ ] Message without file sends and gets AI response -- [ ] Message with file attachment sends and gets AI response -- [ ] No errors in gateway logs during the above flows - ---- - -## File Summary for Phase 1 - -| File | Action | -|------|--------| -| `modules/services/__init__.py` | Replace `ChatService` import with `getService("chat", _ctx, legacy_hub=self)` | -| (No other changes) | Service center ChatService and resolver already support legacy fallback | - ---- - -## FAQ - -**Q: Why start with Chat instead of Utils?** -A: Chat has a clear UI path (chatbot) and only a few call sites. Utils is used everywhere; migrating it later reduces risk. - -**Q: What if `getService("chat", ctx)` fails?** -A: The resolver passes `legacy_hub=self`, so it falls back to the legacy `Services.chat` if the service center module fails to load. You get graceful degradation. - -**Q: Can I test without the frontend?** -A: Yes. Use the API directly, e.g. `POST /api/chatbot/{instanceId}/start/stream` with a valid `UserInputRequest` (with `listFileId` for file upload). diff --git a/docs/SERVICE_CENTER_VS_LEGACY_COMPARISON.md b/docs/SERVICE_CENTER_VS_LEGACY_COMPARISON.md deleted file mode 100644 index d017503e..00000000 --- a/docs/SERVICE_CENTER_VS_LEGACY_COMPARISON.md +++ /dev/null @@ -1,92 +0,0 @@ -# Service Center vs Legacy Services Hub — Comparison & Assessment - -## Executive Summary - -The **Service Center** (`modules/serviceCenter`) is a superior architecture compared to the legacy **Services Hub** (`modules/services`). It was worthwhile to create it. The main benefits are: **explicit dependency graph**, **lazy loading**, **per-service RBAC**, and **context-scoped resolution** without carrying the entire hub. The legacy hub remains valid for incremental migration and backward compatibility. - ---- - -## 1. Architecture Comparison - -| Aspect | Service Center | Legacy Services Hub | -|--------|----------------|---------------------| -| **Location** | `modules/serviceCenter/` | `modules/services/` | -| **Entry point** | `getService(key, context, legacy_hub)` | `getInterface(user, ...)` → `Services` | -| **Constructor** | `(context, get_service)` | `(services)` — full hub | -| **Dependencies** | Declared in registry, resolved lazily via `get_service("key")` | Via `self.services.` — all services always present | -| **Loading** | **Lazy** — only requested services + deps | **Eager** — everything at construction | -| **RBAC** | Per-service `objectKey`, `can_access_service()` | Shared via hub `.rbac` | -| **Caching** | Per-context cache (user + mandate + featureInstance) | No instance cache — new `Services` each call | -| **Feature override** | N/A — features use `getService` directly | Feature services override hub attributes | -| **Pre-warm** | `preWarm()` at app startup | None | -| **Structure** | Core vs importable split; explicit registry | Flat `serviceX/` dirs; discovery via glob | - ---- - -## 2. Which Setup is Better? - -**Service Center is better** for these reasons: - -### 2.1 Explicit Dependency Graph -- Dependencies are declared in `registry.py` (e.g. `"ai": {"dependencies": ["chat", "utils", "extraction", "billing"]}`). -- Circular dependencies are detected and raise `RuntimeError`. -- Easier to reason about and refactor. - -### 2.2 Lazy Loading & Resource Efficiency -- Only requested services (and their transitive deps) are loaded. -- A feature like chatbot needs `chat`, `ai`, `billing`, `streaming` — not `sharepoint`, `ticket`, `neutralization`, etc. -- Legacy hub loads **everything** on first `getInterface()`. - -### 2.3 Context-Scoped Resolution -- Each request gets a `ServiceCenterContext` (user, mandate_id, feature_instance_id, workflow). -- Resolution is cached per context. Same user+mandate+feature → same instances. -- No need to pass or construct a full hub. - -### 2.4 Per-Service RBAC -- Services have `objectKey` (e.g. `service.ai`, `service.extraction`). -- `can_access_service(user, rbac, service_key)` checks before resolving. -- Finer-grained control than a single hub-level RBAC. - -### 2.5 Separation of Concerns -- **Core services** (utils, security, streaming): internal, no RBAC. -- **Importable services** (ai, billing, extraction, etc.): feature-facing, RBAC-protected. -- Clear distinction vs. flat structure in legacy. - -### 2.6 Pre-warm for Cold Start -- `preWarm()` imports all service modules at startup. -- First request avoids import latency. -- Legacy has no equivalent. - ---- - -## 3. When Legacy Still Makes Sense - -- **Migration**: Features that haven’t moved yet still use `getInterface()`. -- **Feature overrides**: Feature-specific services (e.g. `serviceAi/mainServiceAi.py` in a feature) that override hub attributes. -- **Backward compatibility**: `legacy_hub` fallback in Service Center allows gradual migration. - ---- - -## 4. Did It Make Sense to Create the Service Center? - -**Yes.** The legacy hub has inherent limitations: - -1. **Monolithic hub** — every `getInterface()` constructs a full `Services` object with all services, interfaces, and feature discovery. -2. **Implicit dependencies** — services grab what they need via `self.services.`, leading to hidden coupling. -3. **No explicit RBAC per service** — access control is at the hub level. -4. **Eager loading** — every request pays for all services even when only a few are used. - -Service Center addresses these while keeping a migration path via `legacy_hub` fallback. The Chatbot feature already uses it successfully. - ---- - -## 5. Benchmark Script - -Run the comparison script to measure runtime and memory: - -```bash -# From gateway root -python tests/benchmarks/benchmark_service_center_vs_legacy.py -``` - -See `tests/benchmarks/benchmark_service_center_vs_legacy.py` for details on metrics and methodology. diff --git a/tests/unit/serviceCenter/README.md b/tests/unit/serviceCenter/README.md deleted file mode 100644 index fc694519..00000000 --- a/tests/unit/serviceCenter/README.md +++ /dev/null @@ -1,25 +0,0 @@ -# Service Center Tests - -Tests for the Service Center architecture (core/importable services). - -## Run with pytest - -From gateway root (with venv activated): - -```bash -pytest tests/unit/serviceCenter/ -v -``` - -## Standalone smoke test (no pytest) - -```bash -python tests/scripts/smoke_test_service_center.py -``` - -## Test files - -| File | Purpose | -|------|---------| -| `test_service_center_imports.py` | Verify imports of foundation, core, and importable services | -| `test_service_center_resolution.py` | Verify `getService()` resolves services and caches correctly | -| `test_service_center_functionality.py` | Verify basic behavior of UtilsService and other migrated services | diff --git a/tests/unit/serviceCenter/__init__.py b/tests/unit/serviceCenter/__init__.py deleted file mode 100644 index 0a383d21..00000000 --- a/tests/unit/serviceCenter/__init__.py +++ /dev/null @@ -1,2 +0,0 @@ -# Copyright (c) 2025 Patrick Motsch -# Service Center unit tests. diff --git a/tests/unit/serviceCenter/test_service_center_functionality.py b/tests/unit/serviceCenter/test_service_center_functionality.py deleted file mode 100644 index d5498a82..00000000 --- a/tests/unit/serviceCenter/test_service_center_functionality.py +++ /dev/null @@ -1,243 +0,0 @@ -#!/usr/bin/env python3 -# Copyright (c) 2025 Patrick Motsch -# All rights reserved. -""" -Test script: verify basic functionality of migrated services. -Run: pytest gateway/tests/unit/serviceCenter/test_service_center_functionality.py -v -""" - -import pytest -from modules.datamodels.datamodelUam import User -from modules.serviceCenter import getService -from modules.serviceCenter.context import ServiceCenterContext - - -def _make_test_context(): - """Create minimal context for service tests.""" - user = User(id="test-user", username="testuser", email="test@example.com") - return ServiceCenterContext( - user=user, - mandate_id="test-mandate", - feature_instance_id=None, - workflow_id=None, - workflow=None, - ) - - -# ========== UtilsService ========== - -class TestUtilsService: - """Tests for UtilsService.""" - - @pytest.fixture - def utils(self): - ctx = _make_test_context() - return getService("utils", ctx) - - def test_json_strip_code_fences(self, utils): - """stripCodeFences removes markdown code fences.""" - text = '```json\n{"a": 1}\n```' - result = utils.jsonStripCodeFences(text) - assert "```" not in result - assert "{" in result and "}" in result - - def test_json_extract_first_balanced(self, utils): - """extractFirstBalancedJson extracts first balanced JSON.""" - text = 'prefix {"key": "value"} suffix' - result = utils.jsonExtractFirstBalanced(text) - assert "key" in result - assert "value" in result - - def test_json_normalize_text(self, utils): - """normalizeJsonText normalizes JSON string content.""" - text = ' {"a":1} ' - result = utils.jsonNormalizeText(text) - assert result.strip() != "" or text.strip() == "" - - def test_json_try_parse_valid(self, utils): - """tryParseJson parses valid JSON. Returns (obj, error, cleaned_str).""" - obj, err, cleaned = utils.jsonTryParse('{"x": 42}') - assert obj is not None - assert obj.get("x") == 42 - assert err is None - - def test_json_try_parse_invalid(self, utils): - """tryParseJson rejects invalid JSON. Returns (None, error, cleaned_str).""" - obj, err, cleaned = utils.jsonTryParse('{invalid') - assert obj is None - assert err is not None - - def test_sanitize_prompt_content_empty(self, utils): - """sanitizePromptContent returns empty for empty input.""" - assert utils.sanitizePromptContent("") == "" - assert utils.sanitizePromptContent(None) == "" - - def test_sanitize_prompt_content_text(self, utils): - """sanitizePromptContent escapes special chars for text.""" - result = utils.sanitizePromptContent('hello "world"') - assert "world" in result - assert "\\" in result or "'" in result - - def test_sanitize_prompt_content_userinput(self, utils): - """sanitizePromptContent wraps userinput in quotes.""" - result = utils.sanitizePromptContent("test", contentType="userinput") - assert result.startswith("'") and result.endswith("'") - - def test_timestamp_get_utc(self, utils): - """timestampGetUtc returns positive float.""" - ts = utils.timestampGetUtc() - assert isinstance(ts, (int, float)) - assert ts > 0 - - def test_config_get_default(self, utils): - """configGet returns default for missing key.""" - val = utils.configGet("nonexistent_key_xyz", default="fallback") - assert val == "fallback" - - -# ========== TicketService ========== - -class TestTicketService: - """Tests for TicketService (structure only - connectTicket needs async + mocks).""" - - @pytest.fixture - def ticket(self): - ctx = _make_test_context() - return getService("ticket", ctx) - - def test_ticket_has_connect_method(self, ticket): - """TicketService has connectTicket method.""" - assert hasattr(ticket, "connectTicket") - assert callable(getattr(ticket, "connectTicket")) - - -# ========== SharepointService ========== - -class TestSharepointService: - """Tests for SharepointService (pure/sync methods, no Graph API).""" - - @pytest.fixture - def sharepoint(self): - ctx = _make_test_context() - return getService("sharepoint", ctx) - - def test_extract_site_from_standard_path(self, sharepoint): - """extractSiteFromStandardPath parses Microsoft-standard path.""" - result = sharepoint.extractSiteFromStandardPath("/sites/company-share/Docs/Work") - assert result is not None - assert result["siteName"] == "company-share" - assert result["innerPath"] == "Docs/Work" - - def test_extract_site_invalid_path(self, sharepoint): - """extractSiteFromStandardPath returns None for invalid path.""" - assert sharepoint.extractSiteFromStandardPath("invalid") is None - assert sharepoint.extractSiteFromStandardPath("/other/prefix") is None - - def test_validate_path_query(self, sharepoint): - """validatePathQuery validates path format.""" - valid, err = sharepoint.validatePathQuery("/sites/mysite/Documents") - assert valid is True - assert err is None - - valid2, err2 = sharepoint.validatePathQuery("") - assert valid2 is False - assert "empty" in (err2 or "").lower() or err2 is not None - - def test_filter_sites_by_hint(self, sharepoint): - """filterSitesByHint filters by substring.""" - sites = [{"displayName": "Company", "webUrl": "https://a.com"}, {"displayName": "Other", "webUrl": "https://b.com"}] - filtered = sharepoint.filterSitesByHint(sites, "company") - assert len(filtered) == 1 - assert filtered[0]["displayName"] == "Company" - - def test_detect_folder_type(self, sharepoint): - """detectFolderType identifies folders vs files.""" - assert sharepoint.detectFolderType({"folder": {}}) is True - assert sharepoint.detectFolderType({"file": {"mimeType": "pdf"}}) is False - - -# ========== ChatService ========== - -class TestChatService: - """Tests for ChatService (methods that don't require DB/workflow).""" - - @pytest.fixture - def chat(self): - ctx = _make_test_context() - return getService("chat", ctx) - - def test_calculate_object_size(self, chat): - """calculateObjectSize returns byte count.""" - assert chat.calculateObjectSize({"x": 1}) > 0 - assert chat.calculateObjectSize(None) == 0 - - def test_get_workflow_context_no_workflow(self, chat): - """getWorkflowContext returns defaults when no workflow.""" - ctx = chat.getWorkflowContext() - assert ctx["currentRound"] == 0 - assert "currentTask" in ctx - - def test_get_document_reference_from_chat_document(self, chat): - """getDocumentReferenceFromChatDocument produces docItem format.""" - mock_doc = type("Doc", (), {"id": "id-1", "fileName": "f.pdf"})() - ref = chat.getDocumentReferenceFromChatDocument(mock_doc) - assert ref.startswith("docItem:") - assert "id-1" in ref and "f.pdf" in ref - - def test_get_document_count_no_workflow(self, chat): - """getDocumentCount returns message when no workflow.""" - msg = chat.getDocumentCount() - assert "No documents" in msg or "document" in msg.lower() - - -# ========== ExtractionService ========== - -class TestExtractionService: - """Tests for ExtractionService (pure methods, no documents).""" - - @pytest.fixture - def extraction(self): - ctx = _make_test_context() - return getService("extraction", ctx) - - def test_merge_part_results_empty(self, extraction): - """mergePartResults with empty list returns empty string.""" - assert extraction.mergePartResults([]) == "" - - def test_is_json_extraction_response(self, extraction): - """_isJsonExtractionResponse detects extraction format.""" - from modules.datamodels.datamodelExtraction import ContentPart - part = ContentPart(id="p1", label="l1", typeGroup="text", mimeType="text/plain", data='{"extracted_content": {"text": "x", "tables": []}}') - assert extraction._isJsonExtractionResponse([part]) is True - - def test_create_error_response(self, extraction): - """_createErrorResponse produces correct AiCallResponse.""" - err = extraction._createErrorResponse("msg", 10, 20) - assert err.content == "msg" - assert err.errorCount == 1 - - -# ========== WebService ========== - -class TestWebService: - """Tests for WebService (structure only - performWebResearch needs full stack).""" - - @pytest.fixture - def web(self): - try: - ctx = _make_test_context() - return getService("web", ctx) - except (KeyError, ImportError, ModuleNotFoundError): - pytest.skip("WebService dependencies not fully migrated") - - def test_web_has_perform_web_research(self, web): - """WebService has performWebResearch method.""" - assert hasattr(web, "performWebResearch") - assert callable(getattr(web, "performWebResearch")) - - def test_web_workflow_id_property(self, web): - """WebService _workflow_id returns string when no workflow.""" - # No workflow in context -> returns no-workflow- - wf_id = web._workflow_id() - assert isinstance(wf_id, str) - assert "workflow" in wf_id.lower() or wf_id.startswith("no-workflow") diff --git a/tests/unit/serviceCenter/test_service_center_imports.py b/tests/unit/serviceCenter/test_service_center_imports.py deleted file mode 100644 index 9ec79174..00000000 --- a/tests/unit/serviceCenter/test_service_center_imports.py +++ /dev/null @@ -1,126 +0,0 @@ -#!/usr/bin/env python3 -# Copyright (c) 2025 Patrick Motsch -# All rights reserved. -""" -Test script: verify all Service Center modules and migrated services can be imported. -Run: pytest gateway/tests/unit/serviceCenter/test_service_center_imports.py -v -""" - -import pytest - - -# ========== Foundation ========== - -def test_import_service_center_init(): - """Service center main package imports.""" - from modules.serviceCenter import ( - getService, - preWarm, - registerServiceObjects, - can_access_service, - ServiceCenterContext, - CORE_SERVICES, - IMPORTABLE_SERVICES, - SERVICE_RBAC_OBJECTS, - ) - assert getService is not None - assert preWarm is not None - assert registerServiceObjects is not None - assert can_access_service is not None - - -def test_import_service_center_context(): - """Service center context module.""" - from modules.serviceCenter.context import ServiceCenterContext - assert ServiceCenterContext is not None - - -def test_import_service_center_registry(): - """Service center registry with CORE and IMPORTABLE services.""" - from modules.serviceCenter.registry import CORE_SERVICES, IMPORTABLE_SERVICES, SERVICE_RBAC_OBJECTS - assert "utils" in CORE_SERVICES - assert "security" in CORE_SERVICES - assert "streaming" in CORE_SERVICES - assert "web" in IMPORTABLE_SERVICES - assert "ticket" in IMPORTABLE_SERVICES - assert len(SERVICE_RBAC_OBJECTS) > 0 - - -def test_import_service_center_resolver(): - """Service center resolver module.""" - from modules.serviceCenter.resolver import resolve, get_resolution_cache, clear_cache - assert resolve is not None - assert get_resolution_cache is not None - assert clear_cache is not None - - -# ========== Core services ========== - -def test_import_core_utils_service(): - """Core UtilsService can be imported.""" - from modules.serviceCenter.core.serviceUtils.mainServiceUtils import UtilsService - assert UtilsService is not None - - -def test_import_core_security_service(): - """Core SecurityService can be imported.""" - from modules.serviceCenter.core.serviceSecurity.mainServiceSecurity import SecurityService - assert SecurityService is not None - - -def test_import_core_streaming_service(): - """Core StreamingService can be imported.""" - from modules.serviceCenter.core.serviceStreaming.mainServiceStreaming import StreamingService - assert StreamingService is not None - - -# ========== Importable services ========== - -def test_import_service_ticket(): - """Importable TicketService can be imported.""" - from modules.serviceCenter.services.serviceTicket.mainServiceTicket import TicketService - assert TicketService is not None - - -def test_import_service_web(): - """Importable WebService can be imported.""" - from modules.serviceCenter.services.serviceWeb.mainServiceWeb import WebService - assert WebService is not None - - -def test_import_service_sharepoint(): - """Importable SharepointService can be imported.""" - from modules.serviceCenter.services.serviceSharepoint.mainServiceSharepoint import SharepointService - assert SharepointService is not None - - -def test_import_service_chat(): - """Importable ChatService can be imported.""" - from modules.serviceCenter.services.serviceChat.mainServiceChat import ChatService - assert ChatService is not None - - -def test_import_service_extraction(): - """Importable ExtractionService can be imported.""" - from modules.serviceCenter.services.serviceExtraction.mainServiceExtraction import ExtractionService - assert ExtractionService is not None - - -# ========== Optional: services that may still live in legacy hub ========== - -@pytest.mark.parametrize("module_path,class_name", [ - ("modules.serviceCenter.services.serviceMessaging.mainServiceMessaging", "MessagingService"), - ("modules.serviceCenter.services.serviceBilling.mainServiceBilling", "BillingService"), - ("modules.serviceCenter.services.serviceGeneration.mainServiceGeneration", "GenerationService"), - ("modules.serviceCenter.services.serviceAi.mainServiceAi", "AiService"), - ("modules.serviceCenter.services.serviceNeutralization.mainServiceNeutralization", "NeutralizationService"), -]) -def test_import_optional_services(module_path, class_name): - """Services listed in registry - may fail if not yet migrated to serviceCenter.""" - import importlib - try: - mod = importlib.import_module(module_path) - cls = getattr(mod, class_name) - assert cls is not None - except (ImportError, ModuleNotFoundError, AttributeError): - pytest.skip(f"Service {class_name} not yet migrated to serviceCenter") diff --git a/tests/unit/serviceCenter/test_service_center_resolution.py b/tests/unit/serviceCenter/test_service_center_resolution.py deleted file mode 100644 index 7a40363f..00000000 --- a/tests/unit/serviceCenter/test_service_center_resolution.py +++ /dev/null @@ -1,138 +0,0 @@ -#!/usr/bin/env python3 -# Copyright (c) 2025 Patrick Motsch -# All rights reserved. -""" -Test script: verify getService resolves core and importable services correctly. -Run: pytest gateway/tests/unit/serviceCenter/test_service_center_resolution.py -v -""" - -import pytest -from modules.datamodels.datamodelUam import User -from modules.serviceCenter import getService, clear_cache -from modules.serviceCenter.context import ServiceCenterContext - - -def _make_test_context(): - """Create a minimal ServiceCenterContext for tests.""" - user = User(id="test-user", username="testuser", email="test@example.com") - return ServiceCenterContext( - user=user, - mandate_id="test-mandate", - feature_instance_id="test-fi", - workflow_id=None, - workflow=None, - ) - - -@pytest.fixture(autouse=True) -def clear_resolution_cache(): - """Clear resolution cache between tests to avoid cross-test pollution.""" - clear_cache() - yield - clear_cache() - - -# ========== Core services ========== - -def test_resolve_utils(): - """getService('utils') returns UtilsService instance.""" - context = _make_test_context() - svc = getService("utils", context) - assert svc is not None - assert hasattr(svc, "configGet") - assert hasattr(svc, "jsonStripCodeFences") - assert hasattr(svc, "sanitizePromptContent") - - -def test_resolve_security(): - """getService('security') returns SecurityService instance.""" - context = _make_test_context() - svc = getService("security", context) - assert svc is not None - assert hasattr(svc, "_context") - - -def test_resolve_streaming(): - """getService('streaming') returns StreamingService instance.""" - context = _make_test_context() - svc = getService("streaming", context) - assert svc is not None - assert hasattr(svc, "_context") - - -# ========== Importable services (migrated to serviceCenter) ========== - -def test_resolve_ticket(): - """getService('ticket') returns TicketService instance.""" - context = _make_test_context() - svc = getService("ticket", context) - assert svc is not None - assert hasattr(svc, "connectTicket") - - -def test_resolve_sharepoint(): - """getService('sharepoint') returns SharepointService instance (depends on security).""" - context = _make_test_context() - svc = getService("sharepoint", context) - assert svc is not None - assert hasattr(svc, "extractSiteFromStandardPath") - assert hasattr(svc, "setAccessTokenFromConnection") - - -def test_resolve_chat(): - """getService('chat') returns ChatService instance (depends on utils).""" - context = _make_test_context() - svc = getService("chat", context) - assert svc is not None - assert hasattr(svc, "getUserConnectionFromConnectionReference") - assert hasattr(svc, "calculateObjectSize") - - -def test_resolve_extraction(): - """getService('extraction') returns ExtractionService instance (depends on chat, utils).""" - context = _make_test_context() - svc = getService("extraction", context) - assert svc is not None - assert hasattr(svc, "extractContent") - assert hasattr(svc, "mergePartResults") - - -def test_resolve_web(): - """getService('web') returns WebService instance (has ai, chat, utils deps).""" - context = _make_test_context() - # Web depends on ai, chat, utils - may need legacy_hub if ai/chat not migrated - try: - svc = getService("web", context) - assert svc is not None - assert hasattr(svc, "performWebResearch") - except (KeyError, ImportError, ModuleNotFoundError): - pytest.skip("WebService depends on ai/chat which may not be in serviceCenter yet") - - -# ========== Caching ========== - -def test_resolution_is_cached(): - """Same context + key returns same instance.""" - context = _make_test_context() - svc1 = getService("utils", context) - svc2 = getService("utils", context) - assert svc1 is svc2 - - -def test_different_contexts_different_instances(): - """Different contexts produce different instances (different cache keys).""" - ctx1 = _make_test_context() - user2 = User(id="other-user", username="other", email="other@example.com") - ctx2 = ServiceCenterContext(user=user2, mandate_id="m2", feature_instance_id=None) - svc1 = getService("utils", ctx1) - svc2 = getService("utils", ctx2) - assert svc1 is not svc2 - - -# ========== Unknown service ========== - -def test_unknown_service_raises(): - """getService with unknown key raises KeyError.""" - context = _make_test_context() - with pytest.raises(KeyError, match="Unknown service"): - getService("nonexistent", context)