wiki/implementation/SharePoint/concept_sharepoint_feature.md

632 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Concept: SharePoint Site Creator as Gateway Feature
> Implementation concept for integrating the SharePoint Site Creator (site creation + landing page customization) as a Plug&Play feature in the PowerOn gateway platform.
---
## 1. Overview
The SharePoint feature automates two workflows:
1. **Site Creation** — Create a Microsoft 365 Group (which provisions a SharePoint Team Site), set up a folder structure, register the site in a central Kundenmandate list, and apply a branded homepage.
2. **Landing Page Customization** — Compose a page layout from building blocks (text, headers, images, columns, document library previews) and apply it to an existing site's homepage.
Both workflows are exposed as a single gateway feature (`sharepoint`) with two UI views and corresponding API endpoints, following the existing Plug&Play feature pattern.
---
## 2. Feature Identity
| Property | Value |
|----------|-------|
| Feature code | `sharepoint` |
| Feature folder | `modules/features/sharepoint/` |
| Router prefix | `/api/sharepoint` |
| Feature label | `{"en": "SharePoint", "de": "SharePoint", "fr": "SharePoint"}` |
| Feature icon | `mdi-microsoft-sharepoint` (or `mdi-web`) |
---
## 3. File Structure
Following the established gateway convention (same structure as chatbotV2, realEstate, etc.):
```
modules/features/sharepoint/
├── __init__.py
├── mainSharepoint.py # FEATURE_CODE, UI/RESOURCE_OBJECTS, TEMPLATE_ROLES, registration
├── routeFeatureSharepoint.py # APIRouter(prefix="/api/sharepoint"), route handlers
├── interfaceFeatureSharepoint.py # DB access layer (getInterface factory)
├── datamodelFeatureSharepoint.py # Pydantic models: SharepointSiteOrder, LandingPageJob
├── serviceSharepoint.py # Business logic orchestration (create site flow, customize page flow)
├── config.py # SharePointSettings (Pydantic), token cache
└── bridges/
├── __init__.py
├── graphApi.py # Microsoft Graph API client (auth, groups, sites, folders, lists, users)
└── pageCustomization.py # Landing page application (PnP subprocess or SP REST API)
```
No changes to `app.py` or `registry.py` are needed — the registry auto-discovers any folder under `modules/features/` that contains a `routeFeature*.py` file.
**Required one-time changes outside the feature folder:**
| File | Change |
|------|--------|
| `modules/routes/routeSystem.py``_getFeatureUiObjects()` | Add `elif featureCode == "sharepoint"` branch |
| `env_int.env` / `env_prod.env` | Add SharePoint environment variables (see Section 9) |
| Frontend: `pageRegistry.tsx` | Add icon mappings for `feature.sharepoint`, `page.feature.sharepoint.*` |
| Frontend: new view components | `SharepointCreateSiteView.tsx`, `SharepointLandingPageView.tsx` |
| Frontend: new API module | `sharepointApi.ts` |
| Frontend: new hook | `useSharepoint.ts` |
---
## 4. Main Module (`mainSharepoint.py`)
Defines the feature's identity, RBAC catalog objects, and template roles.
### 4.1 UI Objects
Two views — one for creating sites, one for customizing landing pages:
```python
FEATURE_CODE = "sharepoint"
FEATURE_LABEL = {"en": "SharePoint", "de": "SharePoint", "fr": "SharePoint"}
FEATURE_ICON = "mdi-microsoft-sharepoint"
UI_OBJECTS = [
{
"objectKey": "ui.feature.sharepoint.createsite",
"label": {"en": "Create Site", "de": "Site erstellen", "fr": "Créer un site"},
"meta": {"area": "createsite"}
},
{
"objectKey": "ui.feature.sharepoint.landingpage",
"label": {"en": "Landing Page", "de": "Startseite", "fr": "Page d'accueil"},
"meta": {"area": "landingpage"}
},
]
```
### 4.2 Resource Objects
```python
RESOURCE_OBJECTS = [
{
"objectKey": "resource.feature.sharepoint.createSite",
"label": {"en": "Create SharePoint Site", "de": "SharePoint Site erstellen"},
"meta": {"endpoint": "/api/sharepoint/{instanceId}/create-site", "method": "POST"}
},
{
"objectKey": "resource.feature.sharepoint.customizeLandingPage",
"label": {"en": "Customize Landing Page", "de": "Startseite anpassen"},
"meta": {"endpoint": "/api/sharepoint/{instanceId}/customize-landing-page", "method": "POST"}
},
{
"objectKey": "resource.feature.sharepoint.getSiteStatus",
"label": {"en": "Get Site Status", "de": "Site-Status abrufen"},
"meta": {"endpoint": "/api/sharepoint/{instanceId}/site-status/{jobId}", "method": "GET"}
},
{
"objectKey": "resource.feature.sharepoint.listOrders",
"label": {"en": "List Site Orders", "de": "Site-Bestellungen auflisten"},
"meta": {"endpoint": "/api/sharepoint/{instanceId}/orders", "method": "GET"}
},
]
```
### 4.3 Template Roles
```python
TEMPLATE_ROLES = [
{
"roleLabel": "sharepoint-viewer",
"description": {"en": "View site orders (read-only)", "de": "Site-Bestellungen ansehen (nur lesen)"},
"accessRules": [
{"context": "UI", "item": "ui.feature.sharepoint.createsite", "view": True},
{"context": "RESOURCE", "item": "resource.feature.sharepoint.listOrders", "view": True},
{"context": "RESOURCE", "item": "resource.feature.sharepoint.getSiteStatus", "view": True},
{"context": "DATA", "item": None, "view": True, "read": "m", "create": "n", "update": "n", "delete": "n"},
]
},
{
"roleLabel": "sharepoint-user",
"description": {"en": "Create sites and customize landing pages", "de": "Sites erstellen und Startseiten anpassen"},
"accessRules": [
{"context": "UI", "item": None, "view": True},
{"context": "RESOURCE", "item": None, "view": True},
{"context": "DATA", "item": None, "view": True, "read": "m", "create": "m", "update": "m", "delete": "m"},
]
},
{
"roleLabel": "sharepoint-admin",
"description": {"en": "Full access", "de": "Vollzugriff"},
"accessRules": [
{"context": "UI", "item": None, "view": True},
{"context": "RESOURCE", "item": None, "view": True},
{"context": "DATA", "item": None, "view": True, "read": "a", "create": "a", "update": "a", "delete": "a"},
]
},
]
```
The `registerFeature()` and `_syncTemplateRolesToDb()` functions follow the exact same pattern as `mainChatbotV2.py`.
---
## 5. Data Models (`datamodelFeatureSharepoint.py`)
### 5.1 SharepointSiteOrder
Tracks each site creation request (stored in the gateway DB for audit/history):
```python
class SharepointSiteOrder(BaseModel):
id: Optional[str] = None
featureInstanceId: str
mandateId: str
createdBy: str
createdAt: Optional[str] = None
# Form input
projektTitle: str
kurzbeschrieb: str
mandatsId: str
firmenkuerzel: str
klassifizierung: str # intern | vertraulich | geheim
accountManager: str # email
projektLeiter: str # email
projektstart: str # ISO date
projektende: str # ISO date
budget: str
# Result (filled after creation)
status: str = "pending" # pending | provisioning | completed | failed
siteUrl: Optional[str] = None
groupId: Optional[str] = None
siteId: Optional[str] = None
warnings: list[str] = []
errorMessage: Optional[str] = None
```
### 5.2 LandingPageJob
Tracks landing page customization requests:
```python
class LandingPageJob(BaseModel):
id: Optional[str] = None
featureInstanceId: str
mandateId: str
createdBy: str
createdAt: Optional[str] = None
siteUrl: str
pageTitle: str
elements: list[dict] # serialized LandingPageElement list
status: str = "pending" # pending | processing | completed | failed
errorMessage: Optional[str] = None
```
### 5.3 RBAC Table Registration
Add to `modules/interfaces/interfaceRbac.py``TABLE_NAMESPACE`:
```python
"SharepointSiteOrder": "feature.sharepoint",
"LandingPageJob": "feature.sharepoint",
```
This ensures `getRecordsetWithRBAC` filters by `featureInstanceId` and applies MY/GROUP/ALL access levels correctly.
---
## 6. API Routes (`routeFeatureSharepoint.py`)
All routes are scoped under `/api/sharepoint/{instanceId}/...` and validate instance access using the same `_validateInstanceAccess` pattern as other features.
### 6.1 Endpoints
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/{instanceId}/create-site` | Submit site creation order (multipart/form-data with header image) |
| `GET` | `/{instanceId}/orders` | List site orders for this instance (paginated, RBAC-filtered) |
| `GET` | `/{instanceId}/orders/{orderId}` | Get status/details of a specific order |
| `POST` | `/{instanceId}/customize-landing-page` | Submit landing page customization (multipart/form-data) |
| `GET` | `/{instanceId}/landing-page-jobs/{jobId}` | Get status of a landing page job |
### 6.2 Create Site Endpoint (detail)
```
POST /api/sharepoint/{instanceId}/create-site
Content-Type: multipart/form-data
Fields:
projekt_title: str
kurzbeschrieb: str
mandats_id: str
firmenkuerzel: str
klassifizierung: str
account_manager: str (email)
projekt_leiter: str (email)
projektstart: str (ISO date)
projektende: str (ISO date)
budget: str
header_image: File (image)
Response (202 Accepted):
{
"orderId": "uuid",
"status": "provisioning",
"message": "Site creation started. Poll GET /orders/{orderId} for status."
}
```
The 202 response is intentional: site creation takes 3060+ seconds (M365 group provisioning), so the endpoint starts the process asynchronously and returns immediately. The client polls the order status endpoint or (future) uses SSE.
### 6.3 Customize Landing Page Endpoint (detail)
```
POST /api/sharepoint/{instanceId}/customize-landing-page
Content-Type: multipart/form-data
Fields:
site_url: str
page_title: str
elements: str (JSON array of element objects)
header_image: File (optional)
image_0, image_1, ...: File (content images, indexed to match elements array)
Response (202 Accepted):
{
"jobId": "uuid",
"status": "processing",
"message": "Landing page customization started."
}
```
---
## 7. Service Layer (`serviceSharepoint.py`)
Orchestrates the multi-step site creation and page customization workflows.
### 7.1 Site Creation Flow
```
async def createSite(user, mandateId, instanceId, formData, headerImage) -> SharepointSiteOrder:
1. Create SharepointSiteOrder record in DB (status="provisioning")
2. Generate site alias from firmenkuerzel + projektTitle
3. graphApi.getToken()
4. graphApi.createM365Group(alias, title, description)
5. graphApi.pollForSiteReady(groupId, timeout=60s, interval=5s)
6. graphApi.createFolderStructure(siteId, projektTitle) → warning on failure
7. graphApi.addKundenmandateEntry(siteId, formFields) → warning on failure
8. pageCustomization.applyHomepageBanner(siteUrl, title, desc, headerImage) → warning on failure
9. Update order record: status="completed", siteUrl=..., warnings=[...]
10. Return order
```
Steps 68 use the **partial success pattern**: if they fail, warnings are recorded but the order is still marked as completed (the site itself exists and is usable). Only steps 45 (group creation and site provisioning) are critical failures.
### 7.2 Landing Page Customization Flow
```
async def customizeLandingPage(user, mandateId, instanceId, siteUrl, pageTitle, elements, images) -> LandingPageJob:
1. Create LandingPageJob record in DB (status="processing")
2. Upload content images to temp storage
3. pageCustomization.applyDynamicPage(siteUrl, pageTitle, elements, images)
4. Update job record: status="completed" or status="failed"
5. Return job
```
### 7.3 Background Execution
Since both workflows are long-running (3060s for site creation, 1030s for page customization), they should run as `asyncio.create_task()` background tasks. The route handler creates the DB record, starts the task, and returns the order/job ID immediately.
```python
@router.post("/{instanceId}/create-site", status_code=202)
async def create_site(request: Request, instanceId: str, ...):
mandateId = _validateInstanceAccess(instanceId, context)
order = _createOrderRecord(...)
asyncio.create_task(_executeSiteCreation(order))
return {"orderId": order.id, "status": "provisioning"}
```
---
## 8. Bridges
### 8.1 Microsoft Graph API Bridge (`bridges/graphApi.py`)
Encapsulates all Graph API calls with token caching and retry logic.
```python
class GraphApiBridge:
def __init__(self, tenantId, clientId, clientSecret, tenantName):
self._token: str | None = None
self._tokenExpiry: float = 0
...
async def getToken(self) -> str: ...
async def createM365Group(self, alias, displayName, description) -> str: ...
async def pollForSiteReady(self, groupId, timeout=60, interval=5) -> dict: ...
async def getSiteDrives(self, siteId) -> list: ...
async def createFolder(self, driveId, parentId, name) -> dict: ...
async def createFolderStructure(self, siteId, projektTitle) -> dict: ...
async def resolveUserByEmail(self, email) -> dict: ...
async def getBestellportalSite(self) -> dict: ...
async def findKundenmandateList(self, siteId) -> str: ...
async def addKundenmandateEntry(self, fields) -> dict: ...
```
Uses `httpx.AsyncClient` for async HTTP. Implements:
- Token caching with 5-minute pre-expiry refresh
- Exponential backoff retry for 429/503 responses
- Configurable timeout for site provisioning polling
### 8.2 Page Customization Bridge (`bridges/pageCustomization.py`)
Two implementation options (configurable):
**Option A: PnP PowerShell with certificate auth** (recommended for MVP)
```python
class PnpPageBridge:
async def applyHomepageBanner(self, siteUrl, title, subtitle, headerImagePath): ...
async def applyDynamicPage(self, siteUrl, pageTitle, elementsJson, imageDir): ...
```
Invokes PowerShell scripts via `asyncio.create_subprocess_exec`. Uses certificate-based auth (no interactive login). Requires PnP.PowerShell on the server.
**Option B: SharePoint REST API** (recommended for production, no PowerShell dependency)
```python
class SpRestPageBridge:
async def applyHomepageBanner(self, siteUrl, title, subtitle, headerImagePath): ...
async def applyDynamicPage(self, siteUrl, pageTitle, elements, images): ...
```
Uses the SharePoint `/_api/sitepages/pages` REST endpoints directly from Python via `httpx`. More complex to implement but eliminates the PowerShell runtime dependency.
**Decision:** Start with Option A for faster delivery, plan migration to Option B.
---
## 9. Configuration
### 9.1 Environment Variables
Add to `env_int.env` and `env_prod.env`:
```env
# SharePoint Feature - Azure AD App Registration
Feature_Sharepoint_TENANT_ID = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Feature_Sharepoint_CLIENT_ID = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Feature_Sharepoint_CLIENT_SECRET = INT_ENC:...
# SharePoint Feature - Tenant Info
Feature_Sharepoint_TENANT_NAME = contoso.sharepoint.com
Feature_Sharepoint_BASE_URL = https://contoso.sharepoint.com
# SharePoint Feature - Bestellportal
Feature_Sharepoint_BESTELLPORTAL_SITE_PATH = /sites/Bestellportal
Feature_Sharepoint_KUNDENMANDATE_LIST_NAME = Kundenmandat Bestellungen
# SharePoint Feature - Provisioning
Feature_Sharepoint_PROVISIONING_TIMEOUT = 60
Feature_Sharepoint_PROVISIONING_POLL_INTERVAL = 5
# SharePoint Feature - PnP Auth (if using PowerShell bridge)
Feature_Sharepoint_PNP_CERT_PATH = /path/to/cert.pfx
Feature_Sharepoint_PNP_CERT_PASSWORD_SECRET = INT_ENC:...
```
### 9.2 Config Class (`config.py`)
```python
import os
class SharepointConfig:
def __init__(self):
self.tenantId = os.getenv("Feature_Sharepoint_TENANT_ID", "")
self.clientId = os.getenv("Feature_Sharepoint_CLIENT_ID", "")
self.clientSecret = os.getenv("Feature_Sharepoint_CLIENT_SECRET", "")
self.tenantName = os.getenv("Feature_Sharepoint_TENANT_NAME", "")
self.baseUrl = os.getenv("Feature_Sharepoint_BASE_URL", "")
self.bestellportalSitePath = os.getenv("Feature_Sharepoint_BESTELLPORTAL_SITE_PATH", "/sites/Bestellportal")
self.kundenmandateListName = os.getenv("Feature_Sharepoint_KUNDENMANDATE_LIST_NAME", "Kundenmandat Bestellungen")
self.provisioningTimeout = int(os.getenv("Feature_Sharepoint_PROVISIONING_TIMEOUT", "60"))
self.provisioningPollInterval = int(os.getenv("Feature_Sharepoint_PROVISIONING_POLL_INTERVAL", "5"))
_config = None
def getSharepointConfig() -> SharepointConfig:
global _config
if _config is None:
_config = SharepointConfig()
return _config
```
Follows the same naming convention as existing env vars (`Feature_<Name>_<KEY>`).
---
## 10. Frontend Integration
### 10.1 New Files
| File | Purpose |
|------|---------|
| `src/api/sharepointApi.ts` | API client (`createSite`, `customizeLandingPage`, `getOrders`, `getOrderStatus`) |
| `src/hooks/useSharepoint.ts` | React hooks for state management and polling |
| `src/pages/views/sharepoint/SharepointCreateSiteView.tsx` | Site creation form |
| `src/pages/views/sharepoint/SharepointLandingPageView.tsx` | Landing page editor |
| `src/pages/views/sharepoint/SharepointViews.module.css` | Styles |
| `src/pages/views/sharepoint/index.ts` | View exports |
### 10.2 Page Registry Updates
In `src/config/pageRegistry.tsx`, add:
```typescript
// Feature pages - SharePoint
'page.feature.sharepoint.createsite': <FaBuilding />,
'page.feature.sharepoint.landingpage': <FaFileAlt />,
// Feature icon
'feature.sharepoint': <FaBuilding />,
```
### 10.3 FeatureView Mapping
The `FeatureView.tsx` component maps `uiComponent` codes to React components. Add the sharepoint views following the same pattern as chatbotV2 or trustee views.
### 10.4 Create Site Form
The form collects all fields from the SharePoint documentation (Section 5.4):
- `projektTitle`, `kurzbeschrieb`, `mandatsId`, `firmenkuerzel`
- `klassifizierung` (dropdown: intern/vertraulich/geheim)
- `accountManager`, `projektLeiter` (email inputs)
- `projektstart`, `projektende` (date pickers)
- `budget` (text)
- `headerImage` (file upload with preview)
On submit, sends `multipart/form-data` to `POST /api/sharepoint/{instanceId}/create-site`.
Shows a progress indicator after submission, polling `GET /orders/{orderId}` every 3 seconds until status is `completed` or `failed`. Displays the site URL on success and any warnings.
### 10.5 Landing Page Editor
- Input: existing site URL, page title
- Optional header image upload
- Element list with add/remove/reorder (drag-and-drop with `@dnd-kit/core`)
- Supported element types: text, header, image, columns, files
- Each element has a card with type-specific inputs
- Preview of element order before submission
---
## 11. Integration Touchpoints
### 11.1 Navigation (`routeSystem.py`)
Add to `_getFeatureUiObjects()`:
```python
elif featureCode == "sharepoint":
from modules.features.sharepoint.mainSharepoint import UI_OBJECTS
return UI_OBJECTS
```
This enables the navigation API to build menu entries for SharePoint instances.
### 11.2 RBAC
The feature follows the standard RBAC pattern:
- Template roles (`sharepoint-viewer`, `sharepoint-user`, `sharepoint-admin`) are synced to DB on startup
- When an admin creates a SharePoint feature instance for a mandate, template roles are copied
- Route handlers call `_validateInstanceAccess()` before processing
- DB queries use `getRecordsetWithRBAC()` to enforce data-level permissions
### 11.3 Feature Instance Configuration
When an admin creates a feature instance, the `config` JSON field on the `FeatureInstance` can store instance-specific overrides (e.g., different folder structure template, different Bestellportal path). The service layer reads these from the instance config and falls back to environment variables.
---
## 12. Error Handling
Following the partial success pattern from the SharePoint documentation:
```
Create M365 Group → CRITICAL (fail the order)
Poll for Site Ready → CRITICAL (fail with timeout)
Create Folder Structure → NON-CRITICAL (add warning, continue)
Add Kundenmandate Entry → NON-CRITICAL (add warning, continue)
Customize Homepage → NON-CRITICAL (add warning, continue)
```
The `SharepointSiteOrder.warnings` array collects non-critical failure messages. The API response includes these so the frontend can display them.
For transient Graph API errors (429 rate limiting, 503), the bridge implements exponential backoff with up to 3 retries.
---
## 13. Implementation Phases
### Phase 1 — MVP (Core Site Creation)
1. Feature skeleton: `mainSharepoint.py`, `routeFeatureSharepoint.py`, `config.py`, `__init__.py`
2. Data models and interface
3. Graph API bridge: token management, group creation, site polling, folder creation
4. `POST /create-site` endpoint (without Kundenmandate list and page customization)
5. `GET /orders` and `GET /orders/{orderId}` endpoints
6. Frontend: create site form + status polling
7. Navigation and RBAC integration
**Deliverable:** Users can create SharePoint sites with folder structures from the platform.
### Phase 2 — Kundenmandate & Homepage
1. Graph API bridge: user lookup, list operations, Kundenmandate entry creation
2. PnP bridge: homepage banner application (certificate auth)
3. Extend create-site flow with steps 78
4. Frontend: display warnings from partial success
**Deliverable:** Full site creation flow including audit list entry and branded homepage.
### Phase 3 — Landing Page Editor
1. PnP bridge: dynamic page application
2. `POST /customize-landing-page` and `GET /landing-page-jobs/{jobId}` endpoints
3. Data model: `LandingPageJob`
4. Frontend: landing page editor with drag-and-drop, image upload, element types
**Deliverable:** Full feature as described in the SharePoint documentation.
### Phase 4 — Hardening
1. Replace PnP PowerShell with SharePoint REST API (Option B)
2. SSE streaming for real-time progress updates during site creation
3. Idempotency check (verify group alias doesn't already exist before creating)
4. Configurable folder structure templates per feature instance
5. Landing page templates (pre-built layouts users can choose from)
---
## 14. Dependencies
### Backend
| Package | Purpose | Notes |
|---------|---------|-------|
| `httpx` | Async HTTP client for Graph API | Already in the project |
| `python-multipart` | Form data parsing | Already in the project (FastAPI file uploads) |
| PnP.PowerShell (system) | Page customization | Only if using Option A; installed on server OS |
No new Python packages required for the MVP. The Graph API communication uses `httpx` which is already a project dependency.
### Frontend
| Package | Purpose | Notes |
|---------|---------|-------|
| `@dnd-kit/core` | Drag-and-drop for landing page editor | Phase 3 only; evaluate if already available |
### Azure AD
Requires a dedicated App Registration with the permissions listed in the SharePoint documentation Section 3.3 (`Group.ReadWrite.All`, `Sites.FullControl.All`, `Sites.Manage.All`, `User.Read.All`). This can reuse the existing Microsoft service connection (`Service_MSFT_*`) if the required permissions are added, or use a separate registration with `Feature_Sharepoint_*` credentials.
---
## 15. Open Questions
| # | Question | Impact |
|---|----------|--------|
| 1 | Reuse existing `Service_MSFT_*` credentials or create a dedicated app registration for SharePoint? | Config approach, permission scope |
| 2 | Is the Bestellportal site/list already in production, or does it need to be created? | Determines whether Kundenmandate integration is testable from day one |
| 3 | Should the folder structure be hardcoded (Arbeitsdokumente/Ergebnisse/Grundlagendokumente) or configurable per instance? | Affects Phase 1 scope |
| 4 | Is PnP PowerShell available on the Azure App Service, or should we skip page customization initially? | Determines whether Phase 2 homepage branding is feasible |
| 5 | Should site creation progress be streamed via SSE (like chatbot), or is polling sufficient for MVP? | Frontend complexity |