### Gateway Development Framework: Connectors → Interfaces → Services → Workflows

This document explains the gateway's code logic and development framework to build market customer journey features. It focuses on how connectors, interfaces, and services compose a standardized services landscape that workflows and agent models can consume to perform tasks and actions.

---

### Purpose

- **Unify external tools**: Combine many third‑party APIs and utilities behind a consistent interface.
- **Standardize service design**: Model capabilities as reusable services with clear contracts.
- **Enable workflow automation**: Let agent models orchestrate multi‑step tasks using the centralized services.

---

### High‑Level Architecture

1. **Connectors**: Adapters for specific external tools/platforms (auth, transport, retries, mapping).
2. **Interfaces**: Normalization layer exposing common contracts independent of any single vendor.
3. **Services**: Business‑level capabilities built on interfaces, composed into feature‑ready functions.
4. **Service Center**: Central registry/factory to discover, configure, and call services consistently.
5. **Workflows & Agents**: Orchestrations that call services to perform tasks/actions in a workflow engine.

Data/control flow: Client or workflow → Service Center → Service → Interface → Connector → External Tool

---

### Directory Overview (gateway)

- `gateway/modules/connectors/`: Vendor‑specific adapters (HTTP clients, SDK wrappers, auth, mapping).
- `gateway/modules/interfaces/`: Stable contracts and DTOs decoupling services from vendors.
- `gateway/modules/services/`: Business capabilities built from interfaces (e.g., AI, search, files, messaging).
- `gateway/modules/workflows/`: Methods/engines that orchestrate tasks and call services; agent models live here.
- `gateway/modules/routes/`: HTTP endpoints exposing service and workflow capabilities.
- `gateway/modules/security/`: Key management, encryption, and role-based access across services.
- `gateway/modules/shared/`: Cross-cutting utilities (logging, config, error handling, typing).

---

### 1) Connectors: Many External Tools, One Adapter Shape

- **Role**: Provide the lowest-level integration with external systems (API clients, SDKs, auth, retries).
- **Responsibility**:
  - Authentication and credential handling
  - Transport (HTTP/WS), backoff/retry, circuit breaking
  - Response normalization to a minimal internal shape
- **Output**: Vendor‑flavored data mapped to connector models, not directly used by workflows.

Key guidelines:
- Keep connectors vendor‑specific and replaceable.
- No business logic; only integration concerns and basic mapping.

---

### 2) Interfaces: Stable Contracts Over Connectors

- **Role**: Define capability‑oriented contracts (e.g., `ChatModel`, `WebSearch`, `FileStore`) and map connector outputs into interface DTOs.
- **Responsibility**:
  - Normalize differing vendors into consistent methods and data shapes
  - Hide vendor peculiarities behind clear, typed DTOs
  - Offer capability toggles and sensible defaults
- **Output**: Clean, stable methods used by services (e.g., `sendMessage`, `search`, `uploadFile`).

Key guidelines:
- Prefer capability names over vendor names.
- Keep interfaces small, cohesive, and testable with mocks.

---

### 3) Services: Business‑Level Capabilities

- **Role**: Compose one or more interfaces to implement feature‑ready operations (e.g., "answer question with web grounding").
- **Responsibility**:
  - Apply business rules, guardrails, validation, transformations
  - Orchestrate multiple interfaces/connectors when needed
  - Emit domain events/metrics and enforce security policies
- **Output**: High‑level operations that workflows can call atomically.

Key guidelines:
- Services depend on interfaces, not connectors.
- Keep input/output DTOs explicit and versioned when necessary.

---

### 4) Centralized Service Center

- **Role**: A registry/factory that instantiates and exposes services with consistent configuration and lifecycle.
- **Responsibility**:
  - Discoverability: list/get services by capability key
  - Configuration: environment, credentials, routing to specific vendors
  - Cross‑cutting: logging, tracing, caching, rate‑limits, RBAC checks

Usage pattern:
1. Workflow requests a capability (e.g., `ai.answerWithWebGrounding`).
2. Service Center resolves the concrete service instance configured for the tenant/env.
3. Workflow calls the service method with typed input; receives typed output.

---

### 5) Workflows & Agent Models

- **Role**: Coordinate tasks and actions by invoking services in sequence/branches/loops.
- **Responsibility**:
  - Maintain execution state and step transitions
  - Choose actions using agent models and policies
  - Handle retries/compensation and record audit logs

Typical pattern:
1. Ingest user intent/context.
2. Plan next action (agent model) or follow a defined state machine.
3. Call one or more services via the Service Center.
4. Persist outputs, update state, decide next step.

---

### Standardized Interface Example (Conceptual)

```text
interface WebSearch {
  search(query: string, topK: number): SearchResult[]
}

interface ChatModel {
  sendMessage(messages: Message[], tools?: ToolDef[]): ModelResponse
}
```

Vendors like Tavily/Bing/SerpAPI implement `WebSearch` through their connectors; LLM providers implement `ChatModel`. Services compose these interfaces.

---

### Example Service Composition (Conceptual)

```text
Service: AnswerWithWebGrounding
Inputs: question, constraints
Steps:
  1) WebSearch.search → urls/snippets
  2) Fetch & summarize sources
  3) ChatModel.sendMessage with citations → final answer
Outputs: answer, citations, provenance
```

---

### Adding a New Capability

1. **Connector**: Add vendor adapter in `connectors/` and map outputs to internal models.
2. **Interface**: Implement/extend capability contract in `interfaces/` using the new connector.
3. **Service**: Compose the interface in `services/` to expose a business‑oriented method.
4. **Register**: Wire into the Service Center with config/env and RBAC.
5. **Expose**: If needed, add a route in `routes/` for external HTTP access.
6. **Use in Workflows**: Call the new service from methods/agents in `workflows/`.

---

### Security & Governance

- **RBAC**: Enforce role‑based access at the Service Center before invoking services.
- **Secrets**: Manage connector credentials centrally; avoid leaking to workflows.
- **Audit**: Record service calls and workflow steps for traceability.
- **Quotas**: Apply rate limits and budgets per tenant/service.

---

### Observability

- Structured logs at connector, interface, and service layers
- Tracing spans across layers and external calls
- Metrics per capability and vendor, with SLOs

---

### Minimal Request Lifecycle

1. Route receives request or workflow triggers an action.
2. Service Center resolves service instance and validates RBAC.
3. Service executes using interfaces; interfaces call connectors.
4. Results propagate back; logs/metrics recorded; workflow advances state.

---

### Benefits

- **Replace vendors without breaking services** (interfaces shield changes).
- **Accelerate feature delivery** (services are reusable building blocks).
- **Improve reliability and security** (centralized policies and observability).
- **Empower workflows/agents** to perform complex tasks with simple, typed calls.

---

### Quick Map to Code (for orientation)

- `gateway/modules/connectors/` → Vendor adapters (e.g., web search, storage, auth)
- `gateway/modules/interfaces/` → Capability contracts (e.g., `interfaceWebModel`, `interfaceAiModel`)
- `gateway/modules/services/` → Composed capabilities (e.g., `serviceAi`)
- `gateway/modules/workflows/` → Orchestrations/agents (e.g., `methods/methodWeb.py`)

This framework is the backbone for market customer journey features: build once as services, reuse everywhere in workflows.

---

### Visuals

#### Layered Architecture

```mermaid
flowchart TB
  subgraph ClientOrWorkflow[Client / Workflow Engine]
    C[Feature or Agent Task]
  end

  subgraph ServiceCenter[Service Center]
    SC[Registry / Factory\nRBAC, Config, Observability]
  end

  subgraph Services[Services]
    S1[AI Service]
    S2[Web Service]
    S3[File Service]
  end

  subgraph Interfaces[Interfaces]
    I1[ChatModel]
    I2[WebSearch]
    I3[FileStore]
  end

  subgraph Connectors[Connectors]
    K1[Tavily]
    K2[Bing]
    K3[S3]
    K4[LLM Provider]
  end

  C --> SC --> S1 & S2 & S3
  S1 --> I1
  S2 --> I2
  S3 --> I3

  I1 --> K4
  I2 --> K1 & K2
  I3 --> K3
```

#### Request / Action Sequence

```mermaid
sequenceDiagram
  participant Client as Client / Workflow
  participant SC as Service Center
  participant S as Service
  participant I as Interface
  participant K as Connector
  participant EXT as External Tool

  Client->>SC: Request capability (e.g., ai.answerWithWebGrounding)
  SC->>SC: RBAC check, resolve config/vendor
  SC->>S: Get service instance
  S->>I: Call normalized method (e.g., search)
  I->>K: Prepare vendor-specific request
  K->>EXT: API/SDK call (auth, retries)
  EXT-->>K: Response
  K-->>I: Map to normalized DTO
  I-->>S: Return normalized result
  S-->>SC: Business output (validated, enriched)
  SC-->>Client: Typed response, telemetry recorded
```

#### Service Center Components

```mermaid
graph LR
  subgraph SC[Service Center]
    REG[Registry]
    CFG[Config / Env]
    SEC[RBAC / Policies]
    OBS[Logs / Traces / Metrics]
    FAC[Factory]
  end

  REG --> FAC
  CFG --> FAC
  SEC --> FAC
  OBS --> FAC

  FAC -->|builds| Svc[(Service Instances)]

  subgraph Layers[Below Services]
    IF[Interfaces]
    CON[Connectors]
  end

  Svc --> IF --> CON
```

#### Workflow State Machine (Conceptual)

```mermaid
stateDiagram-v2
  [*] --> Plan
  Plan: Decide next action (agent or rules)
  Plan --> CallService: needs external capability
  Plan --> Done: no more steps
  CallService: Invoke via Service Center
  CallService --> HandleResult
  HandleResult: Persist, evaluate, log
  HandleResult --> Plan: more work
  HandleResult --> Done: goal achieved
  Done --> [*]
```