# Swiss Topo STAC API Integration Guide

This document provides comprehensive conceptual documentation for integrating the Swiss Federal Office of Topography (Swisstopo) STAC (SpatioTemporal Asset Catalog) API into the Real Estate feature. This integration enables two primary user workflows: creating projects with enriched parcel data, and browsing maps to explore parcels and their project associations.

## Table of Contents

1. [Overview](#overview)
2. [Architecture](#architecture)
3. [User Story 1: Create Project and Connect Parcels](#user-story-1-create-project-and-connect-parcels)
4. [User Story 2: Browse Map and Explore Parcels](#user-story-2-browse-map-and-explore-parcels)
5. [API Routes and Endpoints](#api-routes-and-endpoints)
6. [Implementation Components](#implementation-components)
7. [Data Flow Summary](#data-flow-summary)
8. [Error Handling and Security](#error-handling-and-security)
9. [Testing and Configuration](#testing-and-configuration)
10. [Summary](#summary)

---

## Overview

### Purpose

The STAC API integration enables the Real Estate feature to support two distinct user workflows:

**User Story 1**: Create a new project, select parcels on a map, automatically enrich them with geospatial data, and store everything in the database for project management.

**User Story 2**: Browse a map, click on parcels to fetch their data from STAC, and see whether each parcel is already connected to an existing project or available for new projects.

### STAC API Overview

The Swiss Federal Office of Topography provides a STAC-compliant API that follows open standards for describing geospatial data. The API provides:

- **Collections**: Groups of related geospatial data (e.g., cadastral parcels, zoning maps)
- **Items**: Individual geospatial features (e.g., a specific parcel, a zoning polygon)
- **Assets**: Downloadable files associated with items (e.g., GeoPackage files, PDFs)

### Key STAC Collections for Real Estate

| Collection ID | Description | Use Case |
|--------------|-------------|----------|
| `ch.swisstopo.amtliche-vermessung` | Official cadastral survey | Identify parcel boundaries |
| `ch.swisstopo.swissboundaries3d-gemeinde-flaeche` | Municipality boundaries | Resolve Gemeinde from coordinates |
| `ch.swisstopo.swissboundaries3d-kanton-flaeche` | Canton boundaries | Resolve Kanton from coordinates |
| `ch.are.nutzungsplanung` | Land use planning (zoning) | Extract bauzone, baulinie |
| `ch.bafu.laermbelastung` | Noise pollution zones | Set laermschutzzone |
| `ch.bafu.hochwassergefahrenkarte` | Flood hazard map | Set hochwasserschutzzone |
| `ch.bafu.grundwasserschutz` | Groundwater protection zones | Set grundwasserschutzzone |

---

## Architecture

The integration follows the existing Gateway architecture pattern with clear separation of concerns:

**Frontend Layer**: User interaction components including map widgets, project forms, and parcel exploration views that communicate with the backend via HTTP/REST.

**API Routes Layer**: HTTP endpoint definitions that handle request validation, authentication, and response formatting. New domain-focused endpoints will replace generic table-based routes.

**Feature Logic Layer**: Business rules and orchestration logic that coordinates STAC queries, data transformation, and enrichment processes. This layer contains the core intelligence for processing location selections, fetching parcel data, and checking parcel-project associations.

**Service Layer**: Specialized services including AI-powered document extraction and text processing capabilities that can analyze zoning regulations from PDF documents.

**Interface Layer**: Normalized data access layer providing CRUD operations on Real Estate entities (Projekt, Parzelle, Gemeinde, etc.) with proper access control and mandate filtering.

**Connector Layer**: External API communication layer that handles STAC API interactions, including query construction, response parsing, error handling, and retry logic.

**Data Storage**: PostgreSQL database for persistent storage of project data, and the Swiss Topo STAC API as the external geospatial data source.

---

## User Story 1: Create Project and Connect Parcels

**User Goal**: Create a new construction project, select parcels on a map, automatically enrich them with geospatial data, and store everything in the database for project management.

### Workflow Steps

1. **Create New Project**: User provides project name and mandate identifier. Backend creates Projekt object with initial status "Analyse" and empty collections. ✅ Already implemented

2. **Select Location on Map**: User clicks on map at location of interest. Frontend sends coordinates (LV95/EPSG:2056) to backend. ❌ New endpoint needed

3. **Identify Parcel from STAC**: Backend queries STAC cadastral collection for parcels intersecting the clicked point. Returns parcel geometry, identifier, properties, and assets. ❌ New functionality needed

4. **Create Parzelle and Connect**: Backend converts STAC geometry to internal GeoPolylinie format, creates Parzelle object, and attaches it to Projekt. Parcel becomes part of project data model for enrichment, annotation, and offline access. ❌ New functionality needed

5. **Enrich Administrative Context**: Backend searches STAC for Gemeinde and Kanton boundaries, creates/updates entities, and establishes hierarchy: Parzelle → Gemeinde → Kanton → Land. ❌ New functionality needed

6. **Extract Zoning Information**: Backend searches zoning collection, calculates intersection areas, selects dominant zone by largest overlap, and extracts bauzone and baulinie. ❌ New functionality needed

7. **Identify Protection Zones**: Backend searches noise, flood, and groundwater collections, finds zones with largest overlap, and extracts zone designations (laermschutzzone, hochwasserschutzzone, grundwasserschutzzone). ❌ New functionality needed

8. **Link Source Documents**: Backend extracts assets from STAC items (GeoPackages, PDFs), creates Dokument objects with metadata, and links them to Parzelle for audit trail. ❌ New functionality needed

9. **AI Extraction (Optional)**: Backend filters PDF documents, calls AI service to extract building parameters (AZ, BZ, vollgeschossZahl, gebaeudehoeheMax, regulations), and updates Parzelle with extracted values. ⚠️ Partial - AI service exists, extraction method needs implementation

10. **Save to Database**: All entities (Projekt, Parzelle, Gemeinde, Kanton, Land, Dokument) are saved to PostgreSQL. Frontend receives enriched Parzelle with enrichment status. ❌ New functionality needed

11. **User Continues Planning**: Users can edit project status, add more parcels, upload documents, add notes, override AI values, and update parcel properties. ⚠️ Partial - Update endpoints exist but may need enhancement

---

## User Story 2: Browse Map and Explore Parcels

**User Goal**: Browse a map, click on parcels to fetch their data from STAC, and see whether each parcel is already connected to an existing project or available for new projects.

### Workflow Steps

1. **Open Map Browser**: User navigates to map browsing interface displaying Switzerland map. ❌ New interface needed

2. **Click on Map**: User clicks at any location. Frontend sends GET request with coordinates and optional mandate ID. ❌ New endpoint needed

3. **Query STAC for Parcel**: Backend queries STAC cadastral collection for parcels at coordinates. Returns parcel data if found. ❌ New functionality needed

4. **Check Project Associations**: Backend searches database for Parzelle records matching STAC parcel identifier, retrieves associated Projekte, filters by mandate if provided, and determines status: Connected (list projects) or Available (not connected). ❌ New functionality needed

5. **Fetch Basic Context (Optional)**: Backend performs lightweight enrichment: administrative boundaries (Gemeinde, Kanton), basic zoning, and protection zones. Uses caching for performance. ❌ New functionality needed

6. **Display Parcel Information**: Frontend shows parcel details, geometry on map, project association status with action buttons (View Project or Create New Project), protection zones, and STAC source indication. ❌ New interface needed

7. **Explore Multiple Parcels**: User can continue clicking to explore multiple parcels. Map shows multiple parcels with color-coded association status. ❌ New functionality needed

8. **Connect Parcel to Project (Optional)**: If parcel is available, user can create new project or add to existing project. System validates permissions, fetches full parcel data, creates Parzelle, triggers full enrichment (User Story 1 Steps 5-9), and saves to database. ❌ New functionality needed

### Key Differences from User Story 1

**User Story 2** focuses on **exploration and discovery**:
- No project creation required upfront
- Lightweight data fetching (can be cached)
- Read-only exploration mode
- Project association checking
- Optional connection to projects

**User Story 1** focuses on **project creation and data persistence**:
- Project must exist first
- Full data enrichment and storage
- All data saved to database
- Complete administrative, zoning, and protection zone context
- AI extraction of building rules

---

## API Routes and Endpoints

### Route Migration Overview

With STAC API integration, the route structure is refactored from generic table-based endpoints to domain-focused endpoints that better align with user workflows and business logic.

### Routes to Deprecate

**POST /api/realestate/query**: Direct SQL queries are a security risk. Use `/command` endpoint for natural language queries or specific GET endpoints for structured data retrieval. ⚠️ Deprecate

**POST /api/realestate/table/{table}**: Generic table creation doesn't reflect domain model. Replace with dedicated project and parcel endpoints. ⚠️ Deprecate

### Routes to Keep

**POST /api/realestate/command**: Keep for natural language commands. May need AI prompt updates for STAC workflows. ✅ Keep

**GET /api/realestate/tables**: Keep for frontend schema discovery. ✅ Keep

**GET /api/realestate/table/{table}**: Keep for admin/debugging. Consider adding domain-specific alternatives. ✅ Keep

### New Domain-Focused Routes

#### Project Management

- **POST /api/realestate/projects**: Create new project (replaces `POST /api/realestate/table/Projekt`)
- **GET /api/realestate/projects**: List projects with filtering and pagination
- **GET /api/realestate/projects/{projekt_id}**: Get single project with full details
- **PATCH /api/realestate/projects/{projekt_id}**: Update project properties
- **DELETE /api/realestate/projects/{projekt_id}**: Delete project with access control

#### Parcel Management (User Story 1)

- **POST /api/realestate/projects/{projekt_id}/locations**: Select location and create Parzelle with full enrichment
- **GET /api/realestate/projects/{projekt_id}/parcels**: List all parcels for a project
- **GET /api/realestate/projects/{projekt_id}/parcels/{parzelle_id}**: Get single parcel with full details
- **PATCH /api/realestate/projects/{projekt_id}/parcels/{parzelle_id}**: Update parcel properties
- **DELETE /api/realestate/projects/{projekt_id}/parcels/{parzelle_id}**: Remove parcel from project
- **POST /api/realestate/projects/{projekt_id}/parcels/{parzelle_id}/refresh-context**: Re-enrich parcel with updated STAC data

#### Map Browsing (User Story 2)

- **GET /api/realestate/parcels/explore**: Browse map and fetch parcel data from STAC with project association check
- **POST /api/realestate/projects/{projekt_id}/parcels/connect**: Connect explored parcel to project (triggers full enrichment)

### Migration Strategy

1. **Phase 1**: Add new domain-focused endpoints alongside existing routes
2. **Phase 2**: Update frontend to use new endpoints
3. **Phase 3**: Mark old endpoints as deprecated with warnings
4. **Phase 4**: Remove deprecated endpoints after migration period

---

## Implementation Components

### STAC Connector

**Purpose**: Encapsulate all STAC API interactions following Gateway connector pattern.

**Key Methods**:
- Search parcels by point (cadastral collection)
- Search administrative boundaries (Gemeinde, Kanton)
- Search zoning (land use planning)
- Search protection zones (noise, flood, groundwater)
- Get item assets (downloadable files)

**Features**: Retry logic with exponential backoff, coordinate system transformations, error handling, response transformation to internal data structures.

**Caching Strategy**: Cache administrative boundaries (rarely change), cache map browsing lookups (short TTL), don't cache project creation searches (user-specific), cache collection metadata.

### Feature Logic Extensions

**For User Story 1**:
- Process location selection and create Parzelle
- Enrich parcel with STAC data (orchestrates full enrichment)
- Extract zoning by calculating intersection areas
- Extract protection zones by overlap calculation
- Geometry conversion helpers (STAC GeoJSON ↔ internal GeoPolylinie)

**For User Story 2**:
- Fetch parcel data from STAC (lightweight)
- Check parcel-project associations (database queries)
- Get basic parcel context (quick administrative/zoning lookup)
- Connect parcel to project (transition to User Story 1)

### AI Service Extension

**Purpose**: Extract building rules from zoning PDF documents.

**Process**: Download PDFs, extract text (OCR if needed), use LLM with structured schema to extract AZ, BZ, vollgeschossZahl, gebaeudehoeheMax, and regulation descriptions.

**Status**: ⚠️ Partial - AI service exists, but zoning rule extraction method needs implementation

---

## Data Flow Summary

### User Story 1: Location Selection Flow

```mermaid
sequenceDiagram
    participant User
    participant Frontend
    participant Route
    participant FeatureLogic
    participant STACConnector
    participant STACAPI
    participant Database
    participant AIService

    User->>Frontend: Click on map
    Frontend->>Route: POST /projects/{id}/locations<br/>(coordinates, project_id)
    Route->>Route: Validate CSRF token<br/>Validate coordinates<br/>Check project access
    
    Route->>FeatureLogic: processLocationSelection()
    FeatureLogic->>STACConnector: search_parcels_by_point(x, y)
    STACConnector->>STACAPI: Query cadastral collection<br/>(point geometry)
    STACAPI-->>STACConnector: Parcel items (geometry, properties, assets)
    STACConnector-->>FeatureLogic: Parcel data
    
    FeatureLogic->>FeatureLogic: Convert STAC geometry<br/>to GeoPolylinie
    FeatureLogic->>FeatureLogic: Create Parzelle object
    FeatureLogic->>Database: Attach Parzelle to Projekt
    
    FeatureLogic->>FeatureLogic: Trigger enrichment
    
    FeatureLogic->>STACConnector: search_administrative_boundaries()
    STACConnector->>STACAPI: Query Gemeinde/Kanton collections
    STACAPI-->>STACConnector: Boundary data
    STACConnector-->>FeatureLogic: Administrative boundaries
    FeatureLogic->>Database: Create/update Gemeinde, Kanton, Land
    
    FeatureLogic->>STACConnector: search_zoning()
    STACConnector->>STACAPI: Query zoning collection
    STACAPI-->>STACConnector: Zoning polygons
    STACConnector-->>FeatureLogic: Zoning data
    FeatureLogic->>FeatureLogic: Calculate intersection areas<br/>Select dominant zone
    FeatureLogic->>FeatureLogic: Extract bauzone, baulinie
    
    FeatureLogic->>STACConnector: search_protection_zones()
    STACConnector->>STACAPI: Query noise/flood/groundwater collections
    STACAPI-->>STACConnector: Protection zone data
    STACConnector-->>FeatureLogic: Protection zones
    FeatureLogic->>FeatureLogic: Extract zone designations
    
    FeatureLogic->>FeatureLogic: Extract assets from STAC items
    FeatureLogic->>Database: Create Dokument objects
    
    opt AI Extraction Enabled
        FeatureLogic->>FeatureLogic: Filter PDF documents
        FeatureLogic->>AIService: extractZoningRules(documents, bauzone)
        AIService->>AIService: Download PDFs<br/>Extract text (OCR if needed)
        AIService->>AIService: LLM analysis with structured schema
        AIService-->>FeatureLogic: Extracted values (AZ, BZ, etc.)
        FeatureLogic->>FeatureLogic: Update Parzelle with AI results
        FeatureLogic->>Database: Add Kontext entry
    end
    
    FeatureLogic->>Database: Save all entities<br/>(Projekt, Parzelle, Gemeinde, etc.)
    FeatureLogic-->>Route: Enriched Parzelle + status
    Route-->>Frontend: Response with enriched Parzelle
    Frontend->>User: Display enriched parcel data
```

### User Story 2: Map Browse Flow

```mermaid
sequenceDiagram
    participant User
    participant Frontend
    participant Route
    participant FeatureLogic
    participant STACConnector
    participant STACAPI
    participant Database

    User->>Frontend: Click on map (browse mode)
    Frontend->>Route: GET /parcels/explore<br/>(coordinates, mandate_id?)
    Route->>Route: Validate coordinates<br/>Validate authentication
    
    Route->>FeatureLogic: fetchParcelData()
    FeatureLogic->>STACConnector: search_parcels_by_point(x, y)
    STACConnector->>STACAPI: Query cadastral collection<br/>(point geometry)
    STACAPI-->>STACConnector: Parcel items (geometry, properties)
    STACConnector-->>FeatureLogic: Parcel data
    
    FeatureLogic->>Database: Check parcel-project associations<br/>(search Parzelle by STAC identifier)
    Database-->>FeatureLogic: Matching Parzelle records<br/>with Projekt IDs
    
    alt Parcels Found in Database
        FeatureLogic->>Database: Get Projekt details
        Database-->>FeatureLogic: Projekt information
        FeatureLogic->>FeatureLogic: Status: Connected<br/>(list projects)
    else No Parcels Found
        FeatureLogic->>FeatureLogic: Status: Available
    end
    
    opt Basic Context Requested
        FeatureLogic->>STACConnector: search_administrative_boundaries()<br/>search_zoning()<br/>search_protection_zones()
        STACConnector->>STACAPI: Query collections (cached if possible)
        STACAPI-->>STACConnector: Context data
        STACConnector-->>FeatureLogic: Basic context
    end
    
    FeatureLogic-->>Route: Parcel data + association status + context
    Route-->>Frontend: Response with parcel info
    Frontend->>User: Display parcel details<br/>Show project association status<br/>Show action buttons
    
    opt User Chooses to Connect
        User->>Frontend: Click "Create Project" or "Add to Project"
        Frontend->>Route: POST /projects/{id}/parcels/connect<br/>(parcel_id, project_id)
        Route->>FeatureLogic: connectParcelToProject()
        Note over FeatureLogic: Triggers User Story 1<br/>enrichment process
        FeatureLogic->>Database: Create Parzelle + enrichment
        FeatureLogic-->>Route: Created Parzelle
        Route-->>Frontend: Success response
    end
```

### AI Extraction Flow (User Story 1)

```mermaid
sequenceDiagram
    participant FeatureLogic
    participant Parzelle
    participant AIService
    participant PDFSource
    participant LLM
    participant Database

    Note over FeatureLogic: After STAC enrichment completes
    
    FeatureLogic->>Parzelle: Get dokumente collection
    Parzelle-->>FeatureLogic: List of Dokument objects
    
    FeatureLogic->>FeatureLogic: Filter documents:<br/>- mimeType == "application/pdf"<br/>- dokumentTyp in [BZO, Bauverordnung]
    
    FeatureLogic->>AIService: extractZoningRules(<br/>parzelleId,<br/>documents[],<br/>bauzone)
    
    loop For each PDF document
        AIService->>PDFSource: Download PDF from dokumentReferenz
        PDFSource-->>AIService: PDF file
        
        AIService->>AIService: Extract text content<br/>(OCR if scanned PDF)
        AIService->>AIService: Preprocess text
    end
    
    AIService->>LLM: Analyze text with structured schema<br/>(prompt: extract building parameters)
    LLM->>LLM: Parse zoning regulations
    LLM-->>AIService: Structured extraction:<br/>- az (float)<br/>- bz (float)<br/>- vollgeschossZahl (int)<br/>- gebaeudehoeheMax (float)<br/>- regelnGrenzabstand (array)<br/>- regelnMehrlaengenzuschlag (array)<br/>- regelnMehrhoehenzuschlag (array)
    
    AIService-->>FeatureLogic: Extraction results
    
    FeatureLogic->>FeatureLogic: Parse and validate results
    
    FeatureLogic->>Parzelle: Update fields:<br/>parzelle.az = result.az<br/>parzelle.bz = result.bz<br/>parzelle.vollgeschossZahl = result.vollgeschossZahl<br/>parzelle.gebaeudehoeheMax = result.gebaeudehoeheMax<br/>parzelle.regelnGrenzabstand = result.regelnGrenzabstand<br/>parzelle.regelnMehrlaengenzuschlag = result.regelnMehrlaengenzuschlag<br/>parzelle.regelnMehrhoehenzuschlag = result.regelnMehrhoehenzuschlag
    
    FeatureLogic->>Database: Create Kontext entry:<br/>thema: "Automatisch extrahierte Baukennzahlen"<br/>inhalt: "AZ/BZ etc. wurden aus BZO [Gemeinde] Stand [Date] extrahiert."
    
    FeatureLogic->>Database: Save updated Parzelle
    
    Database-->>FeatureLogic: Confirmation
    
    FeatureLogic-->>FeatureLogic: Return enrichment results<br/>including AI extraction status
```

---

## Error Handling and Security

### Error Handling

**STAC API Errors**: Retry with exponential backoff (3 attempts), validate coordinates (Switzerland bounds), handle no results found (404 for Story 1, informative response for Story 2), continue with available collections if some fail.

**Database Errors**: Transaction management with rollback, validate relationships before saving, handle parcel already connected scenarios.

**AI Extraction Errors**: Skip failed documents, return partial results, don't fail entire enrichment process.

### Security

**CSRF Protection**: All POST/PATCH endpoints require CSRF tokens. GET endpoints validate authentication.

**Input Validation**: Validate coordinate ranges (Switzerland bounds), whitelist coordinate systems (EPSG:2056, EPSG:4326), validate UUID formats.

**Rate Limiting**: 60 requests/minute for location selection, 120 requests/minute for map browsing, 30 requests/minute for context refresh.

**Data Privacy**: STAC queries are read-only, parcel data stored per mandate with access control, mandate filtering for project associations.

**Access Control**: Users must have permission to create/modify projects, project association information filtered by mandate access, parcel connection requires project modification permissions.

---

## Testing and Configuration

### Testing Approach

**Unit Tests**: STAC connector methods with mock responses, feature logic with mocked connector, project association checking with various database states.

**Integration Tests**: End-to-end flows for both user stories, enrichment pipeline, project association scenarios (not connected, one project, multiple projects, different mandates).

**Mock STAC Responses**: Create realistic mock responses for development and testing without external API dependency.

### Configuration Requirements

**Environment Variables**: STAC base URL, timeout settings, retry configuration, caching settings (different for User Story 1 vs User Story 2), map browse cache TTL.

**Configuration Files**: API endpoint URLs, timeout and retry parameters, caching behavior, collection identifiers.

### Dependencies

**Python Packages**: STAC API client, geospatial operations library, GeoJSON support library.

**System Requirements**: Python 3.10+, internet connection for STAC API, PostgreSQL database, optional PostGIS extension.

**External Services**: Swiss Topo STAC API (public, no authentication), AI service infrastructure (existing).

---

## Summary

This integration guide provides a comprehensive conceptual blueprint for integrating the Swiss Topo STAC API into the Real Estate feature. The implementation enables two distinct user workflows:

**User Story 1: Create Project and Connect Parcels**
- Create new projects and select parcels on a map
- Automatic enrichment with administrative, zoning, and protection zone data
- Full data persistence in database
- AI-powered extraction of building rules from documents
- Complete project management workflow

**User Story 2: Browse Map and Explore Parcels**
- Explore parcels on a map without project creation
- Lightweight data fetching from STAC
- Check if parcels are connected to existing projects
- Optional connection to projects with full enrichment

The architecture follows the existing Gateway patterns:
- **Connectors** for external API communication
- **Interfaces** for normalized data access
- **Features** for business logic
- **Routes** for API endpoints

Both user stories share the STAC connector and enrichment logic, but serve different purposes: User Story 1 focuses on project creation and data persistence, while User Story 2 focuses on exploration and discovery before committing to project creation.

The route structure is refactored from generic table-based endpoints to domain-focused endpoints that better align with user workflows, with a phased migration strategy to ensure backward compatibility.

---

**Document Version**: 2.1  
**Last Updated**: 2025-01-28  
**Author**: Gateway Development Team