Dokumentation zur Roadmap
This commit is contained in:
parent
040067f6af
commit
fbb17f1828
18 changed files with 4462 additions and 748 deletions
369
compliance/poweron_sicherheit_und_compliance.md
Normal file
369
compliance/poweron_sicherheit_und_compliance.md
Normal file
|
|
@ -0,0 +1,369 @@
|
|||
# PowerOn Plattform -- Sicherheit und Compliance
|
||||
|
||||
**Stand:** Februar 2026
|
||||
**Zielgruppe:** Entscheidungsträger, Einkauf, Rechtsabteilung, Datenschutzbeauftragte
|
||||
**Klassifizierung:** Kundeninformation
|
||||
|
||||
---
|
||||
|
||||
## 1. Management Summary
|
||||
|
||||
PowerOn ist eine **Multi-Mandanten-KI-Plattform für Unternehmen**, die es Organisationen ermöglicht, KI-gestützte Geschäftsprozesse sicher, datenschutzkonform und mandantengetrennt zu betreiben. Die Plattform richtet sich an mittlere bis grosse Unternehmen in datenschutzsensiblen Branchen wie Finanzwesen, Treuhand, Immobilien und Beratung.
|
||||
|
||||
Dieses Dokument beschreibt die in PowerOn implementierten Sicherheits- und Datenschutzmassnahmen, ordnet diese gängigen Standards zu und benennt transparent bestehende Einschränkungen.
|
||||
|
||||
**Kernaussagen:**
|
||||
|
||||
- **DSGVO-Betroffenenrechte** (Auskunft, Löschung, Datenübertragbarkeit, Berichtigung) sind als Self-Service-Funktionen direkt in der Plattform implementiert.
|
||||
- **Vollständige Mandantentrennung:** Daten eines Mandanten sind unter keinen Umständen für andere Mandanten einsehbar oder zugänglich.
|
||||
- **Rollenbasierte Zugriffskontrolle (RBAC):** Jeder Datenzugriff wird gegen ein mehrstufiges Berechtigungssystem geprüft -- konfigurierbar pro Mandant und Funktionsmodul.
|
||||
- **Verschlüsselung:** Sensible Konfigurationsdaten sind nach Industriestandard verschlüsselt, sämtliche Kommunikation erfolgt über verschlüsselte Verbindungen.
|
||||
- **Audit-Trail:** Alle sicherheitsrelevanten Aktionen werden lückenlos protokolliert und stehen für Compliance-Nachweise zur Verfügung.
|
||||
- **Transparenz bei KI-Nutzung:** Die Plattform dokumentiert offen, welche Daten an KI-Dienste übermittelt werden, und bietet Konfigurationsoptionen für höchste Datenschutzanforderungen.
|
||||
|
||||
---
|
||||
|
||||
## 2. DSGVO-Konformität
|
||||
|
||||
PowerOn implementiert die zentralen Betroffenenrechte der Datenschutz-Grundverordnung (DSGVO/GDPR) direkt als Plattformfunktionen. Im Folgenden wird für jeden relevanten Artikel beschrieben, was implementiert ist und wo Einschränkungen bestehen.
|
||||
|
||||
### 2.1 Auskunftsrecht (Art. 15 DSGVO)
|
||||
|
||||
Nutzer können über eine Self-Service-Funktion sämtliche über sie gespeicherten Daten exportieren:
|
||||
|
||||
- Persönliche Profildaten (Name, E-Mail, Spracheinstellungen)
|
||||
- Mandatszugehörigkeiten und zugewiesene Rollen
|
||||
- Zugriffsrechte auf Funktionsmodule
|
||||
- Erstellte und eingelöste Einladungen
|
||||
- Zeitpunkte der Kontoerstellung und letzten Anmeldung
|
||||
|
||||
Der Export umfasst alle auf Plattformebene gespeicherten Daten. Feature-spezifische Daten (z.B. Chat-Verläufe, Treuhandpositionen) können über die jeweiligen Funktionsmodule eingesehen werden.
|
||||
|
||||
**Status: Implementiert.**
|
||||
|
||||
### 2.2 Recht auf Löschung (Art. 17 DSGVO)
|
||||
|
||||
Nutzer können ihr Konto und alle zugehörigen Daten eigenständig und unwiderruflich löschen. Dabei gilt:
|
||||
|
||||
- Das System durchsucht automatisch alle Datenbanken (Plattform, Verwaltung, Chat, alle Funktionsmodule) nach Einträgen, die dem Nutzer zugeordnet sind.
|
||||
- Nutzerbezogene Daten werden vollständig gelöscht.
|
||||
- Audit-Logs werden anonymisiert statt gelöscht -- dies gewährleistet die Einhaltung gesetzlicher Aufbewahrungspflichten bei gleichzeitiger Wahrung der Betroffenenrechte.
|
||||
- Die Löschung erfordert eine explizite Bestätigung durch den Nutzer.
|
||||
- Systemadministratoren sind von der Selbstlöschung ausgenommen (Vier-Augen-Prinzip).
|
||||
|
||||
**Status: Implementiert.**
|
||||
|
||||
### 2.3 Recht auf Datenübertragbarkeit (Art. 20 DSGVO)
|
||||
|
||||
Nutzerdaten können in einem maschinenlesbaren, standardisierten Format (JSON-LD nach schema.org) exportiert werden. Dieses Format ermöglicht die Übertragung der Daten an einen anderen Dienstleister.
|
||||
|
||||
**Status: Implementiert.**
|
||||
|
||||
### 2.4 Berichtigungsrecht (Art. 16 DSGVO)
|
||||
|
||||
Nutzer können ihre Profildaten (Name, E-Mail, Spracheinstellungen) jederzeit selbst korrigieren.
|
||||
|
||||
**Status: Implementiert.**
|
||||
|
||||
### 2.5 Transparenz und Informationspflicht
|
||||
|
||||
Die Plattform stellt Nutzern aktiv Informationen über die Datenverarbeitung bereit:
|
||||
|
||||
- Welche Daten erhoben werden
|
||||
- Zu welchem Zweck die Verarbeitung erfolgt
|
||||
- Auf welcher Rechtsgrundlage die Verarbeitung basiert
|
||||
- Welche Aufbewahrungsfristen gelten
|
||||
- Welche Betroffenenrechte bestehen und wie sie ausgeübt werden können
|
||||
|
||||
**Status: Implementiert.**
|
||||
|
||||
### 2.6 Bekannte Einschränkungen
|
||||
|
||||
Die folgenden Punkte sind transparent zu benennen:
|
||||
|
||||
- **Consent-Management:** Die Plattform verfügt derzeit über kein granulares Einwilligungsmanagement mit individuellen Zustimmungs-Toggles. Die Einwilligung zur Datenverarbeitung erfolgt über die Nutzungsbedingungen und, bei Drittanbieter-Authentifizierung (Microsoft, Google), über die jeweiligen OAuth-Einwilligungsflüsse.
|
||||
- **Datenschutz-Kontaktadresse:** Die Kontaktadresse für Datenschutzanfragen ist pro Deployment konfigurierbar und muss vom jeweiligen Betreiber hinterlegt werden.
|
||||
|
||||
---
|
||||
|
||||
## 3. Mandantenmodell und Datenisolation
|
||||
|
||||
### 3.1 Grundprinzip
|
||||
|
||||
PowerOn ist als Multi-Mandanten-Plattform konzipiert. Jede Organisation, Abteilung oder jeder Kunde wird als eigenständiger **Mandant** abgebildet. Das zentrale Sicherheitsversprechen:
|
||||
|
||||
> **Daten eines Mandanten sind unter keinen Umständen für Nutzer anderer Mandanten sichtbar oder zugänglich.**
|
||||
|
||||
### 3.2 Wie die Trennung funktioniert
|
||||
|
||||
- **Zugehörigkeitsprüfung bei jedem Zugriff:** Bevor ein Nutzer auf Mandantendaten zugreifen kann, prüft die Plattform, ob eine aktive Mitgliedschaft des Nutzers in diesem Mandanten besteht. Ohne nachgewiesene Mitgliedschaft wird der Zugriff verweigert.
|
||||
- **Kein mandantenübergreifender Datenfluss:** Datenbankabfragen werden automatisch auf den Mandantenkontext gefiltert. Es gibt keinen Mechanismus, der Daten mandantenübergreifend zusammenführt oder exponiert.
|
||||
- **Feature-Isolation:** Innerhalb eines Mandanten werden Funktionsmodule (z.B. Chatbot, Treuhand, Immobilien) zusätzlich isoliert. Nutzer benötigen für jedes Funktionsmodul eine explizite Zugriffsberechtigung.
|
||||
|
||||
### 3.3 Mehrfachmandanten
|
||||
|
||||
Nutzer können gleichzeitig in mehreren Mandanten arbeiten -- ein häufiges Szenario bei Beratern, Treuhändern oder Dienstleistern mit mehreren Kunden. Die Plattform stellt sicher:
|
||||
|
||||
- Der Mandantenkontext wird pro Anfrage bestimmt, nicht pro Sitzung. Ein Wechsel zwischen Mandanten ist jederzeit möglich, ohne dass Daten vermischt werden.
|
||||
- Es findet keine Übertragung von Daten, Berechtigungen oder Einstellungen zwischen Mandanten statt.
|
||||
|
||||
### 3.4 Schutz vor Manipulation
|
||||
|
||||
Auch bei technischem Wissen über die Plattformarchitektur ist ein unbefugter Zugriff auf fremde Mandantendaten nicht möglich: Die Zugehörigkeitsprüfung erfolgt serverseitig und kann nicht durch Manipulation von Anfrageparametern umgangen werden.
|
||||
|
||||
---
|
||||
|
||||
## 4. Rollenbasierte Zugriffskontrolle (RBAC)
|
||||
|
||||
### 4.1 Berechtigungsmodell
|
||||
|
||||
PowerOn verfügt über ein feingliedriges, rollenbasiertes Berechtigungssystem. Für jede Aktion (Lesen, Erstellen, Bearbeiten, Löschen) können individuelle Berechtigungsstufen vergeben werden:
|
||||
|
||||
| Berechtigungsstufe | Beschreibung |
|
||||
|---|---|
|
||||
| Kein Zugriff | Funktion ist nicht verfügbar |
|
||||
| Eigene Daten | Zugriff nur auf selbst erstellte Einträge |
|
||||
| Mandantendaten | Zugriff auf alle Daten innerhalb des eigenen Mandanten |
|
||||
| Alle Daten | Vollzugriff (typischerweise für Administratoren) |
|
||||
|
||||
### 4.2 Konfigurierbarkeit
|
||||
|
||||
- Rollen können **pro Mandant** und **pro Funktionsmodul** definiert und zugewiesen werden.
|
||||
- Es gibt keine fest verdrahteten Berechtigungen -- jede Organisation kann das Rollenmodell an ihre Bedürfnisse anpassen.
|
||||
- Berechtigungsänderungen werden im Audit-Trail protokolliert.
|
||||
|
||||
### 4.3 Administratoren
|
||||
|
||||
Systemadministratoren verfügen über erweiterte Rechte, unterliegen jedoch ebenfalls dem RBAC-System. Alle Administratoraktionen werden gesondert im Audit-Log festgehalten. Es gibt kein unkontrolliertes "Superuser"-Konto ohne Nachvollziehbarkeit.
|
||||
|
||||
---
|
||||
|
||||
## 5. Verschlüsselung und Datensicherheit
|
||||
|
||||
### 5.1 Verschlüsselung ruhender Daten
|
||||
|
||||
Alle sensiblen Konfigurationsdaten werden verschlüsselt gespeichert:
|
||||
|
||||
- **Verschlüsselungsverfahren:** AES-Verschlüsselung (Fernet)
|
||||
- **Schlüsselableitung:** PBKDF2-HMAC-SHA256 nach aktuellem Industriestandard
|
||||
- **Umfang:** Datenbankpasswörter, API-Schlüssel für KI-Dienste, OAuth-Geheimnisse, JWT-Schlüssel und alle weiteren als "SECRET" gekennzeichneten Konfigurationswerte
|
||||
|
||||
Sensible Konfigurationsdaten liegen zu keinem Zeitpunkt im Klartext in der Konfiguration oder im Quellcode vor.
|
||||
|
||||
### 5.2 Verschlüsselung in Übertragung
|
||||
|
||||
- Sämtliche Kommunikation zwischen Client und Server erfolgt über HTTPS/TLS.
|
||||
- Verbindungen zu Drittdiensten (KI-Anbieter, Authentifizierungsdienste, E-Mail-Dienste) nutzen ebenfalls ausschliesslich verschlüsselte Verbindungen.
|
||||
- Die TLS-Konfiguration erfolgt auf Infrastruktur-Ebene (Azure / Reverse Proxy) -- dies entspricht dem Branchenstandard bei Cloud-Deployments und ermöglicht zentrale Verwaltung und Aktualisierung der Zertifikate.
|
||||
|
||||
### 5.3 Zugriffskontrolle auf Verschlüsselungsschlüssel
|
||||
|
||||
- Der Zugriff auf Verschlüsselungsschlüssel ist auf das Minimum beschränkt.
|
||||
- Jeder Zugriff auf Verschlüsselungsfunktionen (Entschlüsselung, Neuverschlüsselung) wird im Audit-Trail protokolliert.
|
||||
- Eine Ratenbegrenzung schützt vor automatisierten Entschlüsselungsversuchen.
|
||||
|
||||
---
|
||||
|
||||
## 6. Schutzmassnahmen gegen Angriffe
|
||||
|
||||
PowerOn implementiert Schutzmassnahmen gegen die gängigsten Angriffsvektoren für Webanwendungen:
|
||||
|
||||
### 6.1 Cross-Site Request Forgery (CSRF)
|
||||
|
||||
Alle datenverändernden Operationen (Erstellen, Ändern, Löschen) erfordern ein gültiges Sicherheitstoken. Damit wird verhindert, dass schädliche Webseiten im Namen eines eingeloggten Nutzers Aktionen auslösen können.
|
||||
|
||||
### 6.2 Ratenbegrenzung (Rate Limiting)
|
||||
|
||||
Jede API-Funktion ist mit individuellen Zugriffslimits versehen, die automatisierte Angriffe und Missbrauch unterbinden:
|
||||
|
||||
| Funktion | Limit |
|
||||
|---|---|
|
||||
| Anmeldung | 5 Versuche pro Minute |
|
||||
| Datenexport (DSGVO) | 5 Anfragen pro Minute |
|
||||
| Kontolöschung | 1 Anfrage pro Stunde |
|
||||
| Chatbot-Nutzung | 120 Anfragen pro Minute |
|
||||
| Datei-Upload | 10 Uploads pro Minute |
|
||||
|
||||
### 6.3 Eingabebereinigung
|
||||
|
||||
Nutzereingaben werden bereinigt und validiert, bevor sie an KI-Modelle oder Datenbanken weitergeleitet werden. Dies schützt vor Prompt-Injection-Angriffen und anderen Manipulationsversuchen.
|
||||
|
||||
### 6.4 SQL-Injection-Schutz
|
||||
|
||||
- Alle Datenbankabfragen der Plattform verwenden parametrisierte Abfragen -- der Industriestandard zur Vermeidung von SQL-Injection.
|
||||
- Der Chatbot-Datenbankzugriff ist zusätzlich auf reine Leseabfragen (SELECT) beschränkt. Schreibende, ändernde oder löschende Operationen sind auf Systemebene blockiert.
|
||||
|
||||
### 6.5 Cross-Origin Resource Sharing (CORS)
|
||||
|
||||
Nur definierte und verifizierte Quelldomains erhalten Zugriff auf die Plattform-API. Anfragen von nicht autorisierten Quellen werden automatisch abgelehnt.
|
||||
|
||||
---
|
||||
|
||||
## 7. KI-Dienste und Datenverarbeitung
|
||||
|
||||
Transparenz im Umgang mit KI-Diensten ist für datenschutzbewusste Organisationen entscheidend. PowerOn legt offen, wie Daten im Kontext der KI-Nutzung verarbeitet werden.
|
||||
|
||||
### 7.1 Welche Daten werden verarbeitet
|
||||
|
||||
Im Rahmen der KI-gestützten Funktionen (Chatbot, Workflow-Verarbeitung, Dokumentenanalyse) können folgende Daten an KI-Dienste übermittelt werden:
|
||||
|
||||
- Nutzeranfragen und -eingaben
|
||||
- Dokumentinhalte (bei Dokumentenanalyse)
|
||||
- Gesprächsverläufe (bei Chat-Funktionen)
|
||||
|
||||
### 7.2 Welche KI-Anbieter werden genutzt
|
||||
|
||||
PowerOn unterstützt mehrere KI-Anbieter, die je nach Bedarf und Konfiguration eingesetzt werden:
|
||||
|
||||
- **OpenAI** (GPT-4o und weitere Modelle)
|
||||
- **Anthropic** (Claude-Modelle)
|
||||
- **Tavily** (Websuche)
|
||||
- **Private LLM** (lokale/eigene Modelle -- kein externer Datenabfluss)
|
||||
|
||||
Die Auswahl des Anbieters ist konfigurierbar und kann an die Datenschutzanforderungen des Kunden angepasst werden.
|
||||
|
||||
### 7.3 Mandantentrennung bei KI-Anfragen
|
||||
|
||||
Jede KI-Anfrage erfolgt im Kontext des jeweiligen Mandanten. Es findet keine Vermischung von Daten verschiedener Mandanten in KI-Anfragen statt.
|
||||
|
||||
### 7.4 Kein Training mit Kundendaten
|
||||
|
||||
Bei Nutzung der Enterprise-API-Vereinbarungen der KI-Anbieter (OpenAI Enterprise API, Anthropic API) werden Kundendaten nicht für das Training der KI-Modelle verwendet. Dies ist vertraglich durch die Auftragsverarbeitungsvereinbarungen (AV-V / DPA) mit den jeweiligen Anbietern abgesichert.
|
||||
|
||||
### 7.5 Optionen für höchste Datenschutzanforderungen
|
||||
|
||||
Für Organisationen mit besonders hohen Datenschutzanforderungen bietet PowerOn:
|
||||
|
||||
- **Datenschutz-Neutralisierer:** Optionales Modul, das personenbezogene Daten vor der Übermittlung an externe KI-Dienste entfernt oder pseudonymisiert.
|
||||
- **Private-LLM-Anbindung:** Möglichkeit, ein eigenes, lokal betriebenes Sprachmodell zu nutzen. In diesem Fall verlassen keine Daten die eigene Infrastruktur.
|
||||
|
||||
### 7.6 Bekannte Einschränkung
|
||||
|
||||
Die automatische Erkennung und Filterung personenbezogener Daten (PII) vor dem Versand an externe KI-Dienste ist **nicht standardmässig aktiviert**. Organisationen, die mit besonders sensiblen personenbezogenen Daten arbeiten, sollten den Datenschutz-Neutralisierer nutzen oder den Private-LLM-Connector einsetzen.
|
||||
|
||||
---
|
||||
|
||||
## 8. Audit-Trail und Nachvollziehbarkeit
|
||||
|
||||
### 8.1 Was wird protokolliert
|
||||
|
||||
Sämtliche sicherheitsrelevanten Aktionen werden automatisch und lückenlos in einem Audit-Log erfasst:
|
||||
|
||||
| Kategorie | Beispiele |
|
||||
|---|---|
|
||||
| Zugriff | Anmeldungen, fehlgeschlagene Anmeldeversuche, Abmeldungen |
|
||||
| Sicherheit | Administratoraktionen, SysAdmin-Zugriffe, Sicherheitsereignisse |
|
||||
| Datenschutz (DSGVO) | Datenexporte, Kontolöschungen, Portabilitätsanfragen |
|
||||
| Berechtigungen | Rollenzuweisungen, Berechtigungsänderungen |
|
||||
| Verschlüsselung | Zugriffe auf Verschlüsselungsfunktionen |
|
||||
| Datenoperationen | Zugriffe auf sensible Geschäftsdaten |
|
||||
|
||||
### 8.2 Aufbewahrung und Bereinigung
|
||||
|
||||
- **Standard-Aufbewahrungsdauer:** 365 Tage (konfigurierbar)
|
||||
- **Automatische Bereinigung:** Veraltete Einträge werden durch einen täglichen Prozess entfernt -- dies stellt sicher, dass Audit-Daten nicht unbefristet aufbewahrt werden (DSGVO-Konformität).
|
||||
- **Anonymisierung:** Bei der Löschung eines Nutzerkontos werden zugehörige Audit-Einträge anonymisiert statt gelöscht. Die Nachvollziehbarkeit sicherheitsrelevanter Ereignisse bleibt gewahrt, ohne dass Rückschlüsse auf die gelöschte Person möglich sind.
|
||||
|
||||
### 8.3 Nutzung für Compliance-Nachweise
|
||||
|
||||
Die Audit-Daten können als Nachweis für interne und externe Audits herangezogen werden. Sie dokumentieren, wer wann welche sicherheitsrelevante Aktion durchgeführt hat, und unterstützen damit die Anforderungen an die Rechenschaftspflicht nach Art. 5 Abs. 2 DSGVO.
|
||||
|
||||
---
|
||||
|
||||
## 9. Einordnung in gängige Standards
|
||||
|
||||
PowerOn orientiert sich an anerkannten Standards und Rahmenwerken. Die folgende Einordnung beschreibt transparent, welche Anforderungen die Plattform bereits abdeckt und wo Ergänzungen erforderlich sind.
|
||||
|
||||
### 9.1 DSGVO / GDPR
|
||||
|
||||
| Anforderung | Status | Bemerkung |
|
||||
|---|---|---|
|
||||
| Betroffenenrechte (Art. 15--17, 20) | Implementiert | Auskunft, Löschung, Portabilität, Berichtigung als Self-Service |
|
||||
| Rechenschaftspflicht (Art. 5 Abs. 2) | Implementiert | Lückenloser Audit-Trail |
|
||||
| Verzeichnis der Verarbeitungstätigkeiten (Art. 30) | Unterstützt | Audit-Log liefert die Datenbasis; das formale Verzeichnis muss vom Betreiber geführt werden |
|
||||
| Technische und organisatorische Massnahmen (Art. 32) | Implementiert | Verschlüsselung, Zugriffskontrolle, Mandantentrennung, Eingabevalidierung |
|
||||
| Einwilligungsmanagement (Art. 7) | Teilweise | Über Nutzungsbedingungen und OAuth; kein granulares Consent-Tool |
|
||||
|
||||
### 9.2 Schweizer Datenschutzgesetz (nDSG / revDSG)
|
||||
|
||||
Die Anforderungen des revidierten Schweizer Datenschutzgesetzes sind mit den DSGVO-Massnahmen kompatibel. Insbesondere:
|
||||
|
||||
- Informationspflicht bei Datenerhebung: Transparenzfunktion implementiert
|
||||
- Recht auf Datenherausgabe und -löschung: Self-Service-Funktionen vorhanden
|
||||
- Pflicht zu angemessenen technischen Massnahmen: Verschlüsselung, RBAC, Mandantentrennung
|
||||
|
||||
### 9.3 OWASP Top 10
|
||||
|
||||
Die Plattform adressiert die häufigsten Web-Sicherheitsrisiken gemäss OWASP:
|
||||
|
||||
| OWASP-Risiko | Massnahme in PowerOn |
|
||||
|---|---|
|
||||
| Broken Access Control | Rollenbasierte Zugriffskontrolle, Mandantenprüfung bei jedem Zugriff |
|
||||
| Cryptographic Failures | AES-Verschlüsselung, PBKDF2-Schlüsselableitung, HTTPS/TLS |
|
||||
| Injection | Parametrisierte Datenbankabfragen, Eingabebereinigung, SQL-Leseeinschränkung |
|
||||
| Security Misconfiguration | CORS-Einschränkungen, Rate Limiting, CSRF-Schutz |
|
||||
| Identification and Authentication Failures | JWT-basierte Authentifizierung, Token-Widerruf, Ratenbegrenzung bei Anmeldung |
|
||||
|
||||
Es besteht **keine formale OWASP-Zertifizierung**. Die Massnahmen basieren auf den OWASP-Empfehlungen und sind als präventive Sicherheitsmassnahmen implementiert.
|
||||
|
||||
### 9.4 ISO 27001 / BSI IT-Grundschutz
|
||||
|
||||
Die implementierten technischen und organisatorischen Massnahmen (Zugriffskontrolle, Verschlüsselung, Audit-Logging, Eingabevalidierung, Mandantentrennung) bilden eine **solide Grundlage** für ein Informationssicherheits-Managementsystem (ISMS) nach ISO 27001 oder BSI IT-Grundschutz.
|
||||
|
||||
Es besteht **keine formale Zertifizierung**. Die vorhandene Infrastruktur ermöglicht es jedoch, eine Zertifizierung auf dieser Basis gezielt anzustreben.
|
||||
|
||||
---
|
||||
|
||||
## 10. Authentifizierung und Identitätsmanagement
|
||||
|
||||
### 10.1 Anmeldemethoden
|
||||
|
||||
PowerOn unterstützt mehrere Authentifizierungsverfahren:
|
||||
|
||||
- **Lokale Anmeldung:** Benutzername und Passwort mit JWT-basierter Sitzungsverwaltung
|
||||
- **Microsoft-Anmeldung (Azure AD / Entra ID):** Single Sign-On über bestehende Microsoft-Konten
|
||||
- **Google-Anmeldung:** Single Sign-On über Google Workspace
|
||||
|
||||
### 10.2 Sitzungssicherheit
|
||||
|
||||
- Authentifizierungstoken werden in sicheren, HTTP-only Cookies gespeichert (nicht im Browser-Speicher zugänglich)
|
||||
- Tokens haben eine konfigurierbare Gültigkeitsdauer
|
||||
- Token-Widerruf ist jederzeit möglich (z.B. bei Verdacht auf Kompromittierung)
|
||||
- Bei lokaler Anmeldung wird die Gültigkeit des Tokens zusätzlich gegen die Datenbank geprüft
|
||||
|
||||
### 10.3 Automatische Token-Erneuerung
|
||||
|
||||
Authentifizierungstoken werden automatisch erneuert, bevor sie ablaufen. Dies gewährleistet eine unterbrechungsfreie Nutzung bei gleichzeitiger Begrenzung der Token-Gültigkeitsdauer.
|
||||
|
||||
---
|
||||
|
||||
## 11. Offene Punkte und Empfehlungen
|
||||
|
||||
Transparenz schafft Vertrauen. Die folgenden Punkte sind offen benannt, damit Kunden und Betreiber informierte Entscheidungen treffen können.
|
||||
|
||||
| Thema | Status | Empfehlung |
|
||||
|---|---|---|
|
||||
| Granulares Consent-Management | Nicht vorhanden | Falls regulatorisch erforderlich, als separates Modul ergänzen |
|
||||
| PII-Filterung vor KI-Versand | Nicht standardmässig aktiv | Datenschutz-Neutralisierer aktivieren oder Private LLM einsetzen |
|
||||
| Security-Header (CSP, HSTS) | Auf Infrastruktur-Ebene | Konfiguration auf Reverse-Proxy-Ebene dokumentieren und prüfen |
|
||||
| Datenschutz-Kontaktadresse | Platzhalter | Pro Deployment mit tatsächlicher DSB-Kontaktadresse konfigurieren |
|
||||
| Formale Zertifizierungen | Keine vorhanden | ISO 27001 / BSI auf Basis der vorhandenen Massnahmen anstrebbar |
|
||||
|
||||
---
|
||||
|
||||
## 12. Zusammenfassung
|
||||
|
||||
PowerOn vereint Enterprise-KI-Funktionalität mit einem umfassenden Sicherheits- und Datenschutzkonzept. Die Plattform bietet:
|
||||
|
||||
- **DSGVO-konforme Betroffenenrechte** als Self-Service-Funktionen
|
||||
- **Vollständige Mandantentrennung** mit serverseitiger Zugehörigkeitsprüfung
|
||||
- **Feingliedriges Berechtigungssystem** mit individuell konfigurierbaren Rollen
|
||||
- **Verschlüsselung nach Industriestandard** für ruhende und übertragene Daten
|
||||
- **Lückenlosen Audit-Trail** für Compliance-Nachweise
|
||||
- **Transparente KI-Datenverarbeitung** mit Optionen für höchste Datenschutzanforderungen
|
||||
|
||||
Gleichzeitig werden bestehende Einschränkungen offen kommuniziert und Empfehlungen für ergänzende Massnahmen gegeben. Diese Kombination aus implementierter Sicherheit und transparenter Kommunikation bildet die Grundlage für eine vertrauensvolle Zusammenarbeit.
|
||||
|
||||
---
|
||||
|
||||
*Dieses Dokument basiert auf einer Analyse der PowerOn-Plattform (Stand Februar 2026). Alle beschriebenen Massnahmen sind in der Plattform implementiert und wurden anhand der Codebasis verifiziert. Angaben ohne Gewähr -- für verbindliche Zusicherungen gelten die jeweiligen Vertragsvereinbarungen.*
|
||||
347
implementation/Chatbot/FUNCTIONAL_DIFFERENCES_ANALYSIS.md
Normal file
347
implementation/Chatbot/FUNCTIONAL_DIFFERENCES_ANALYSIS.md
Normal file
|
|
@ -0,0 +1,347 @@
|
|||
# Functional Differences Analysis: Legacy vs Current Chatbot
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The **legacy implementation works correctly** because the LLM **actually uses the `send_streaming_message` tool** as instructed. The **current implementation fails** because the LLM **generates status messages as regular text** instead of using the tool, causing an infinite loop when the system tries to handle these text messages.
|
||||
|
||||
---
|
||||
|
||||
## Core Functional Difference
|
||||
|
||||
### Legacy: Tool-Based Status Updates (WORKS)
|
||||
|
||||
**How it works:**
|
||||
1. System prompt instructs: "Use `send_streaming_message` tool for status updates"
|
||||
2. LLM (ChatAnthropic) **follows instructions** and calls the tool
|
||||
3. Event handler listens **ONLY** for `on_tool_start` events with `send_streaming_message`
|
||||
4. When tool is called → routes to tools node → tool executes → routes back to agent
|
||||
5. Agent then calls SQL tools → processes results → generates final answer
|
||||
|
||||
**Code Evidence:**
|
||||
```python
|
||||
# legacy/chatbot.py line 267
|
||||
if etype == "on_tool_start" and ename == "send_streaming_message":
|
||||
tool_in = edata.get("input") or {}
|
||||
msg = tool_in.get("message")
|
||||
if isinstance(msg, str) and msg.strip():
|
||||
yield {"type": "status", "label": msg.strip()}
|
||||
continue
|
||||
```
|
||||
|
||||
**Key Point:** Legacy **ONLY** handles tool calls. It doesn't try to detect status messages in regular text.
|
||||
|
||||
---
|
||||
|
||||
### Current: Text-Based Status Updates (BROKEN)
|
||||
|
||||
**How it fails:**
|
||||
1. System prompt instructs: "MUST use `send_streaming_message` tool, VERBOTEN to write text messages"
|
||||
2. LLM (AICenterChatModel) **ignores instructions** and generates text messages like "Ich werde die Datenbank nach Artikeln durchsuchen..."
|
||||
3. Event handler tries to handle these text messages by detecting them as "status messages"
|
||||
4. When status message detected → routes back to agent (to "fix" it)
|
||||
5. Agent generates another text status message (still not using tool)
|
||||
6. **Infinite loop** until max iterations (15) reached
|
||||
|
||||
**Code Evidence:**
|
||||
```python
|
||||
# gateway/modules/features/chatbot/chatbotStreaming.py line 198-227
|
||||
if etype == "on_chain_stream" and ename == "agent":
|
||||
# Tries to detect status messages in regular text
|
||||
if content and is_status_message(content):
|
||||
await _emit_status_event(...) # Convert to status event
|
||||
continue # Don't store as message
|
||||
```
|
||||
|
||||
```python
|
||||
# gateway/modules/features/chatbot/chatbotLangGraph.py line 292-296
|
||||
if is_status:
|
||||
# Status message without tool calls - route back to agent
|
||||
# The agent should then call actual tools (like sqlite_query)
|
||||
logger.info(f"Status message detected without tool calls, routing back to agent...")
|
||||
return "agent" # THIS CAUSES THE LOOP
|
||||
```
|
||||
|
||||
**Key Point:** Current tries to **compensate** for LLM not following instructions, but this creates a loop.
|
||||
|
||||
---
|
||||
|
||||
## Why They Differ: Root Causes
|
||||
|
||||
### 1. Model Behavior Difference
|
||||
|
||||
| Aspect | Legacy (ChatAnthropic) | Current (AICenterChatModel) |
|
||||
|--------|----------------------|---------------------------|
|
||||
| **Tool Calling** | Follows prompt, uses `send_streaming_message` tool | Ignores prompt, generates text instead |
|
||||
| **Instruction Following** | Strong adherence to system prompt | Weak adherence to system prompt |
|
||||
| **Model Type** | Direct LangChain integration | Bridge to AI center (may use different models) |
|
||||
|
||||
**Impact:** The current model doesn't follow the instruction to use the tool, so it generates text messages that break the workflow.
|
||||
|
||||
---
|
||||
|
||||
### 2. Event Handling Strategy
|
||||
|
||||
#### Legacy Event Handling
|
||||
```python
|
||||
# Simple: Only listen for tool calls
|
||||
if etype == "on_tool_start" and ename == "send_streaming_message":
|
||||
# Handle tool call
|
||||
yield {"type": "status", "label": msg}
|
||||
continue # Done, move on
|
||||
```
|
||||
|
||||
**Strategy:** Trust the LLM to use the tool. Only handle tool calls.
|
||||
|
||||
#### Current Event Handling
|
||||
```python
|
||||
# Complex: Try to handle both tool calls AND text messages
|
||||
if etype == "on_tool_start" and ename == "send_streaming_message":
|
||||
# Handle tool call (same as legacy)
|
||||
await _emit_status_event(...)
|
||||
|
||||
if etype == "on_chain_stream" and ename == "agent":
|
||||
# ALSO try to detect status messages in text
|
||||
if is_status_message(content):
|
||||
await _emit_status_event(...) # Convert text to status
|
||||
```
|
||||
|
||||
**Strategy:** Don't trust the LLM. Try to compensate by detecting status messages in text.
|
||||
|
||||
**Problem:** This creates a feedback loop where status messages trigger re-routing, causing infinite loops.
|
||||
|
||||
---
|
||||
|
||||
### 3. Workflow Routing Logic
|
||||
|
||||
#### Legacy Routing (`should_continue`)
|
||||
```python
|
||||
# Simple logic
|
||||
def should_continue(state: ChatState) -> str:
|
||||
last_message = state.messages[-1]
|
||||
tool_calls = getattr(last_message, "tool_calls", None)
|
||||
|
||||
if tool_calls:
|
||||
return "tools" # Has tool calls → execute tools
|
||||
else:
|
||||
return END # No tool calls → done
|
||||
```
|
||||
|
||||
**Key Point:** No special handling for status messages. If there are tool calls, execute them. Otherwise, end.
|
||||
|
||||
#### Current Routing (`should_continue`)
|
||||
```python
|
||||
# Complex logic with status detection
|
||||
def should_continue(state: ChatState) -> str:
|
||||
last_message = state.messages[-1]
|
||||
tool_calls = getattr(last_message, "tool_calls", None)
|
||||
|
||||
if tool_calls:
|
||||
return "tools"
|
||||
|
||||
# NEW: Check if it's a status message
|
||||
if isinstance(last_message, AIMessage):
|
||||
content = last_message.content
|
||||
if is_status_message(content):
|
||||
return "agent" # Route back to agent (CAUSES LOOP!)
|
||||
|
||||
return END
|
||||
```
|
||||
|
||||
**Key Point:** Tries to "fix" status messages by routing back to agent, but agent just generates another status message.
|
||||
|
||||
---
|
||||
|
||||
### 4. Message Filtering
|
||||
|
||||
#### Legacy: No Filtering
|
||||
- All messages are stored in memory
|
||||
- Status messages from tool calls are handled, but messages themselves are stored
|
||||
- No filtering of "status-like" text messages
|
||||
|
||||
#### Current: Aggressive Filtering
|
||||
```python
|
||||
# chatbotMemory.py - Filters out status messages
|
||||
if content:
|
||||
content_lower = content.lower().strip()
|
||||
status_patterns = ["ich werde", "ich suche", ...]
|
||||
if len(content) < 150 and any(pattern in content_lower for pattern in status_patterns):
|
||||
logger.debug(f"Skipping status update message...")
|
||||
continue # Don't store
|
||||
```
|
||||
|
||||
```python
|
||||
# chatbotLangGraph.py - Filters from conversation window
|
||||
if content and is_status_message(content):
|
||||
logger.debug(f"Filtering out status message from conversation window...")
|
||||
# Skip this message
|
||||
```
|
||||
|
||||
**Problem:** Status messages are filtered out, so they don't accumulate in memory, but the agent keeps generating them, creating a loop.
|
||||
|
||||
---
|
||||
|
||||
## The Infinite Loop Explained
|
||||
|
||||
### What Happens in Current Implementation
|
||||
|
||||
1. **User asks:** "wie viele leds haben wir auf lager"
|
||||
2. **Agent generates:** "Ich werde die Datenbank nach Artikeln durchsuchen..." (text message, NO tool call)
|
||||
3. **Status detection:** `is_status_message()` returns `True`
|
||||
4. **Routing:** `should_continue()` returns `"agent"` (route back)
|
||||
5. **Memory filtering:** Message is filtered out (not stored)
|
||||
6. **Agent called again:** Generates another status message (still no tool call)
|
||||
7. **Repeat steps 3-6** until max iterations (15) reached
|
||||
8. **Workflow ends:** No final answer, only status messages
|
||||
|
||||
### What Should Happen (Like Legacy)
|
||||
|
||||
1. **User asks:** "wie viele leds haben wir auf lager"
|
||||
2. **Agent calls tool:** `send_streaming_message("Durchsuche Datenbank nach LEDs...")` (tool call)
|
||||
3. **Tool execution:** Tool node executes, emits status event
|
||||
4. **Routing:** `should_continue()` returns `"tools"` → tools execute → back to agent
|
||||
5. **Agent calls SQL tool:** `sqlite_query("SELECT ...")` (tool call)
|
||||
6. **SQL execution:** Tool node executes query, returns results
|
||||
7. **Agent processes:** Generates final answer with results
|
||||
8. **Workflow ends:** Final answer returned
|
||||
|
||||
---
|
||||
|
||||
## Why Legacy Works: Model Compliance
|
||||
|
||||
### ChatAnthropic Behavior
|
||||
- **Strong tool calling:** When instructed to use a tool, it actually uses it
|
||||
- **Prompt following:** Adheres to system prompt instructions
|
||||
- **Tool-first approach:** Prefers tool calls over text for structured operations
|
||||
|
||||
### Evidence from Legacy Logs
|
||||
```
|
||||
Denke nach.. ← Tool call
|
||||
Durchsuche Datenbank nach LEDs... ← Tool call
|
||||
Berechne Gesamtlagerbestand... ← Tool call
|
||||
Formuliere finale Antwort... ← Tool call
|
||||
Aus der Datenbank habe ich 801... ← Final text answer
|
||||
```
|
||||
|
||||
Each status update is a **tool call**, not a text message.
|
||||
|
||||
---
|
||||
|
||||
## Why Current Fails: Model Non-Compliance
|
||||
|
||||
### AICenterChatModel Behavior
|
||||
- **Weak tool calling:** Doesn't reliably use tools when instructed
|
||||
- **Text-first approach:** Generates text messages instead of tool calls
|
||||
- **Prompt ignoring:** Doesn't follow "VERBOTEN" instructions
|
||||
|
||||
### Evidence from Current Logs
|
||||
```
|
||||
Ich werde die Datenbank nach Artikeln durchsuchen... ← Text message (WRONG!)
|
||||
Skipping status update message... ← Filtered out
|
||||
Status message detected without tool calls... ← Detected as status
|
||||
Routing back to agent... ← Causes loop
|
||||
[Repeats 15 times]
|
||||
```
|
||||
|
||||
Each status update is a **text message**, not a tool call, causing the loop.
|
||||
|
||||
---
|
||||
|
||||
## System Prompt Comparison
|
||||
|
||||
### Legacy Prompt (Works)
|
||||
```
|
||||
STREAMING-UPDATES: Du hast Zugriff auf das Tool "send_streaming_message",
|
||||
mit dem du dem Nutzer kurze Status-Updates senden kannst.
|
||||
Nutze dieses Tool, um den Nutzer über deine aktuellen Aktivitäten zu informieren.
|
||||
```
|
||||
|
||||
**Tone:** Informative, suggests using the tool.
|
||||
|
||||
### Current Prompt (Doesn't Work)
|
||||
```
|
||||
STREAMING-UPDATES - ABSOLUT KRITISCH:
|
||||
⚠️⚠️⚠️ WICHTIG: Du MUSST das Tool "send_streaming_message" verwenden,
|
||||
um Status-Updates zu senden. VERBOTEN ist es, normale Text-Nachrichten
|
||||
für Status-Updates zu schreiben!
|
||||
|
||||
VERBOTEN: Text-Nachrichten wie "Ich werde die Datenbank durchsuchen..."
|
||||
ERLAUBT: Nur das Tool "send_streaming_message" für Status-Updates verwenden!
|
||||
```
|
||||
|
||||
**Tone:** Aggressive, forbids text messages, but model ignores it anyway.
|
||||
|
||||
**Irony:** The more explicit the prompt, the more the model ignores it.
|
||||
|
||||
---
|
||||
|
||||
## Functional Differences Summary
|
||||
|
||||
| Aspect | Legacy | Current | Impact |
|
||||
|--------|--------|---------|--------|
|
||||
| **Model Tool Calling** | ✅ Uses tool | ❌ Generates text | **CRITICAL** |
|
||||
| **Event Handling** | Tool calls only | Tool calls + text detection | Creates complexity |
|
||||
| **Routing Logic** | Simple (tool calls → tools, else → end) | Complex (status detection → route back) | Creates loop |
|
||||
| **Message Filtering** | None | Aggressive filtering | Hides the problem |
|
||||
| **Prompt Style** | Informative | Aggressive/forbidding | Model ignores anyway |
|
||||
|
||||
---
|
||||
|
||||
## Why This Matters
|
||||
|
||||
### Legacy Success Factors
|
||||
1. **Model compliance:** ChatAnthropic follows instructions
|
||||
2. **Simple event handling:** Only handles what's expected (tool calls)
|
||||
3. **No compensation logic:** Doesn't try to "fix" model behavior
|
||||
4. **Trust-based:** Assumes model will use tools correctly
|
||||
|
||||
### Current Failure Factors
|
||||
1. **Model non-compliance:** AICenterChatModel doesn't follow instructions
|
||||
2. **Complex event handling:** Tries to handle both tool calls and text
|
||||
3. **Compensation logic:** Tries to "fix" model behavior, creates loops
|
||||
4. **Distrust-based:** Assumes model won't use tools, tries to compensate
|
||||
|
||||
---
|
||||
|
||||
## The Real Problem
|
||||
|
||||
The current implementation is trying to **compensate for model non-compliance** by:
|
||||
1. Detecting status messages in text
|
||||
2. Converting them to status events
|
||||
3. Routing back to agent to "fix" it
|
||||
|
||||
But this creates a **feedback loop** because:
|
||||
- Agent generates text status message
|
||||
- System detects it and routes back
|
||||
- Agent generates another text status message
|
||||
- Loop continues
|
||||
|
||||
**The solution is NOT to add more compensation logic.** The solution is to **fix the root cause**: Make the model actually use the tool.
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Short-Term Fix
|
||||
1. **Remove status message detection** from `should_continue()` - don't route back
|
||||
2. **Remove text-to-status conversion** - only handle tool calls
|
||||
3. **Let status messages be stored** - don't filter them aggressively
|
||||
4. **Simplify routing** - if no tool calls, end (like legacy)
|
||||
|
||||
### Long-Term Fix
|
||||
1. **Fix model behavior** - Ensure AICenterChatModel actually uses tools
|
||||
2. **Improve prompt** - Test different prompt styles to get tool usage
|
||||
3. **Model selection** - Use a model that reliably follows tool-calling instructions
|
||||
4. **Tool binding** - Verify tools are properly bound and available to model
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The functional difference is **not in the architecture** but in **model behavior**:
|
||||
|
||||
- **Legacy:** Model uses tools → Simple event handling → Works
|
||||
- **Current:** Model doesn't use tools → Complex compensation → Breaks
|
||||
|
||||
The current implementation is **over-engineered** to compensate for model non-compliance, but this compensation creates more problems than it solves.
|
||||
|
||||
**The fix is simple:** Make the model use the tool (like legacy), then simplify the event handling to match legacy's simplicity.
|
||||
|
|
@ -1,236 +0,0 @@
|
|||
# Streaming Utility Architecture: Event-Driven Real-Time Updates
|
||||
|
||||
## Current Implementation
|
||||
|
||||
### Event Manager (`modules/features/chatbot/eventManager.py`)
|
||||
|
||||
The `StreamingEventManager` is a **generic, reusable** event manager that provides:
|
||||
|
||||
- **Generic Event Queue Management**: Per-context asyncio queues (not just workflows)
|
||||
- **Event Emission**: `emit_event()` method supporting multiple event types and categories
|
||||
- **Event Streaming**: `stream_events()` async generator for SSE streaming
|
||||
- **Automatic Cleanup**: Queue cleanup after delay (60 seconds default)
|
||||
- **Event Categories**: Filtering by category (chat, workflow, document, etc.)
|
||||
- **Thread-Safe**: Lock-based synchronization for concurrent access
|
||||
|
||||
### Architecture Overview
|
||||
|
||||
The streaming system uses a **pure event-driven approach**:
|
||||
|
||||
1. **Event Emission**: When data changes (messages created, logs written), events are emitted directly
|
||||
2. **Event Queue**: Events are queued per context (workflow_id, document_id, etc.)
|
||||
3. **SSE Streaming**: Route endpoint streams events from queue in real-time
|
||||
4. **No Database Polling**: The SSE endpoint does NOT poll the database - it only streams queued events
|
||||
|
||||
### Current Usage: Chatbot Feature
|
||||
|
||||
The chatbot feature (`modules/features/chatbot/`) demonstrates the streaming architecture:
|
||||
|
||||
- **Event Types**: `chatdata`, `complete`, `stopped`, `error`
|
||||
- **Event Categories**: `chat`, `workflow`
|
||||
- **Direct Emission**: Events emitted when messages/logs are created in `mainChatbot.py`
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### 1. Event Manager API (`modules/features/chatbot/eventManager.py`)
|
||||
|
||||
```python
|
||||
class StreamingEventManager:
|
||||
"""
|
||||
Generic event manager for real-time streaming across all features.
|
||||
Supports multiple event types and contexts (workflows, documents, tasks, etc.)
|
||||
Thread-safe event emission and queue management.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._queues: Dict[str, asyncio.Queue] = {}
|
||||
self._locks: Dict[str, asyncio.Lock] = {}
|
||||
self._cleanup_tasks: Dict[str, asyncio.Task] = {}
|
||||
self._subscribers: Dict[str, Set[str]] = {} # context_id -> set of queue_ids
|
||||
|
||||
def create_queue(self, context_id: str) -> asyncio.Queue:
|
||||
"""Create a new event queue for a context"""
|
||||
|
||||
def get_queue(self, context_id: str) -> Optional[asyncio.Queue]:
|
||||
"""Get existing event queue for a context"""
|
||||
|
||||
def has_queue(self, context_id: str) -> bool:
|
||||
"""Check if a queue exists for a context"""
|
||||
|
||||
async def emit_event(
|
||||
self,
|
||||
context_id: str, # workflow_id, document_id, task_id, etc.
|
||||
event_type: str, # "message", "log", "status", "progress", "complete", "error", "chatdata"
|
||||
data: Dict[str, Any], # Flexible data structure
|
||||
event_category: str = "default", # "chat", "workflow", "document", etc.
|
||||
message: Optional[str] = None, # For backward compatibility
|
||||
step: Optional[str] = None # For backward compatibility
|
||||
):
|
||||
"""Emit event to the context's event queue"""
|
||||
|
||||
async def stream_events(
|
||||
self,
|
||||
context_id: str,
|
||||
event_categories: Optional[List[str]] = None,
|
||||
timeout: Optional[float] = None
|
||||
) -> AsyncIterator[Dict[str, Any]]:
|
||||
"""Async generator for streaming events from a context"""
|
||||
|
||||
async def cleanup(self, context_id: str, delay: float = 60.0):
|
||||
"""Schedule cleanup of event queue after delay"""
|
||||
```
|
||||
|
||||
**Global Singleton**: Access via `get_event_manager()` function
|
||||
|
||||
### 2. SSE Route Implementation (`modules/routes/routeChatbot.py`)
|
||||
|
||||
The chatbot streaming endpoint (`/api/chatbot/start/stream`) demonstrates the pattern:
|
||||
|
||||
```python
|
||||
@router.post("/start/stream")
|
||||
async def stream_chatbot_start(...) -> StreamingResponse:
|
||||
event_manager = get_event_manager()
|
||||
|
||||
# Start background processing (creates workflow and event queue)
|
||||
workflow = await chatProcess(currentUser, userInput, workflowId)
|
||||
|
||||
# Get or create event queue
|
||||
queue = event_manager.get_queue(workflow.id) or event_manager.create_queue(workflow.id)
|
||||
|
||||
async def event_stream():
|
||||
"""Pure event-driven streaming (no database polling)"""
|
||||
# 1. Send initial chat data once (from database)
|
||||
chatData = interfaceDbChat.getUnifiedChatData(workflow.id, None)
|
||||
if chatData.get("items"):
|
||||
for item in filtered_items:
|
||||
yield f"data: {json.dumps(item)}\n\n"
|
||||
|
||||
# 2. Stream events from queue (event-driven)
|
||||
while True:
|
||||
try:
|
||||
# Get event from queue (blocks until event available)
|
||||
event = await asyncio.wait_for(queue.get(), timeout=1.0)
|
||||
|
||||
# Handle event types
|
||||
if event["type"] == "chatdata":
|
||||
yield f"data: {json.dumps(event["data"])}\n\n"
|
||||
elif event["type"] == "complete":
|
||||
break # Close stream
|
||||
# ... other event types
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
# Send keepalive every 30 seconds
|
||||
yield f": keepalive\n\n"
|
||||
continue
|
||||
|
||||
return StreamingResponse(
|
||||
event_stream(),
|
||||
media_type="text/event-stream",
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no"
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
**Key Points**:
|
||||
- **Initial Data**: Fetched once from database at stream start
|
||||
- **Event Streaming**: Pure event-driven from queue (no polling)
|
||||
- **Keepalive**: Sent every 30 seconds to keep connection alive
|
||||
- **Status Check**: Periodic workflow status check (every 5 seconds) only for stopped detection
|
||||
|
||||
### 3. Event Emission in Processing Code
|
||||
|
||||
#### A. Chatbot Message Processing (`modules/features/chatbot/mainChatbot.py`)
|
||||
|
||||
Events are emitted **directly when data is created**:
|
||||
|
||||
```python
|
||||
from modules.features.chatbot.eventManager import get_event_manager
|
||||
|
||||
event_manager = get_event_manager()
|
||||
|
||||
# When creating a user message
|
||||
userMessage = interfaceDbChat.createMessage(userMessageData)
|
||||
|
||||
# Emit event immediately (exact chatData format)
|
||||
await event_manager.emit_event(
|
||||
context_id=workflow.id,
|
||||
event_type="chatdata",
|
||||
data={
|
||||
"type": "message",
|
||||
"createdAt": message_timestamp,
|
||||
"item": userMessage.dict()
|
||||
},
|
||||
event_category="chat"
|
||||
)
|
||||
|
||||
# When creating assistant message
|
||||
assistantMessage = interfaceDbChat.createMessage(assistantMessageData)
|
||||
|
||||
# Emit event immediately
|
||||
await event_manager.emit_event(
|
||||
context_id=workflowId,
|
||||
event_type="chatdata",
|
||||
data={
|
||||
"type": "message",
|
||||
"createdAt": message_timestamp,
|
||||
"item": assistantMessage.dict()
|
||||
},
|
||||
event_category="chat"
|
||||
)
|
||||
|
||||
# When workflow completes
|
||||
await event_manager.emit_event(
|
||||
context_id=workflowId,
|
||||
event_type="complete",
|
||||
data={"workflowId": workflowId},
|
||||
event_category="workflow",
|
||||
message="Chatbot-Verarbeitung abgeschlossen",
|
||||
step="complete"
|
||||
)
|
||||
```
|
||||
|
||||
#### B. Log Events
|
||||
|
||||
Logs are stored in database and then emitted as events:
|
||||
|
||||
```python
|
||||
# Store log in database
|
||||
log_data = {
|
||||
"id": f"log_{uuid.uuid4()}",
|
||||
"workflowId": workflowId,
|
||||
"message": "Analysiere Benutzeranfrage...",
|
||||
"type": "info",
|
||||
"timestamp": getUtcTimestamp(),
|
||||
"status": "running",
|
||||
"roundNumber": round_number
|
||||
}
|
||||
interfaceDbChat.createLog(log_data)
|
||||
|
||||
# Note: Logs are emitted via the route's periodic chatData fetch mechanism
|
||||
# OR can be emitted directly as events if needed
|
||||
```
|
||||
|
||||
**Event Format**: Events use the exact `chatData` format: `{type, createdAt, item}`
|
||||
|
||||
## Benefits of Event-Driven Streaming
|
||||
|
||||
### Performance
|
||||
- **Reduced Server Load**: No constant database queries every 0.5-3 seconds
|
||||
- **Lower Latency**: Events delivered immediately (< 100ms) vs polling delay (500-3000ms)
|
||||
- **Bandwidth Efficiency**: Only send data when it changes, not empty responses
|
||||
- **No Database Polling**: SSE endpoint does NOT poll database - pure event-driven
|
||||
|
||||
### User Experience
|
||||
- **Real-time Updates**: Users see progress instantly as events occur
|
||||
- **Better Responsiveness**: No perceived delay from polling intervals
|
||||
- **Reduced Battery**: Mobile devices consume less power without constant polling
|
||||
- **Immediate Feedback**: Messages appear as soon as they're created
|
||||
|
||||
### Scalability
|
||||
- **Horizontal Scaling**: Event queues can be distributed (Redis, RabbitMQ) in future
|
||||
- **Connection Management**: Better handling of many concurrent streams
|
||||
- **Resource Efficiency**: One persistent connection vs many HTTP requests
|
||||
- **Memory Efficient**: Queues cleaned up automatically after workflow completion
|
||||
|
|
@ -1,378 +0,0 @@
|
|||
# Chatbot vs FastTrack: Architecture Comparison
|
||||
|
||||
## Chatbot Basic Implementation
|
||||
|
||||
### Overview
|
||||
|
||||
The chatbot is a specialized feature designed for handling user queries that require database access or web research. It provides immediate responses by returning a workflow object instantly, then processes the request asynchronously in the background.
|
||||
|
||||
### Core Purpose
|
||||
|
||||
The chatbot analyzes user input to determine what database queries or web research are needed, executes them, and generates comprehensive answers based on real data rather than just AI knowledge.
|
||||
|
||||
### Main Entry Point
|
||||
|
||||
**Function**: `modules/features/chatbot/mainChatbot.py::chatProcess()`
|
||||
|
||||
**Signature**:
|
||||
```python
|
||||
async def chatProcess(
|
||||
currentUser: User,
|
||||
userInput: UserInputRequest,
|
||||
workflowId: Optional[str] = None
|
||||
) -> ChatWorkflow
|
||||
```
|
||||
|
||||
### Basic Flow
|
||||
|
||||
1. **Workflow Creation/Resumption**
|
||||
- Creates new workflow or resumes existing one
|
||||
- Generates conversation name from user prompt
|
||||
- Sets workflow mode to `WORKFLOW_CHATBOT`
|
||||
- Creates event queue for streaming
|
||||
|
||||
2. **Message Storage**
|
||||
- Stores user message immediately
|
||||
- Emits message event for streaming
|
||||
- Returns workflow object (instant response to user)
|
||||
|
||||
3. **Background Processing** (async)
|
||||
- Analyzes user input to determine query needs
|
||||
- Generates SQL queries if database access needed
|
||||
- Executes queries in parallel
|
||||
- Performs web research if needed
|
||||
- Generates final answer with all results
|
||||
|
||||
### Key Components
|
||||
|
||||
#### 1. Analysis Phase (`_processChatbotMessage`)
|
||||
|
||||
**Purpose**: Determines what queries/research are needed
|
||||
|
||||
**Implementation**:
|
||||
- Uses `get_initial_analysis_prompt()` from `chatbotConstants.py`
|
||||
- Calls AI via `MethodAi.process()` with `simpleMode=True`
|
||||
- Returns JSON with:
|
||||
- `needsDatabaseQuery`: Boolean
|
||||
- `needsWebResearch`: Boolean
|
||||
- `sqlQueries[]`: Array of SQL query objects
|
||||
- `reasoning`: Explanation of analysis
|
||||
|
||||
**Query Structure**:
|
||||
```json
|
||||
{
|
||||
"query": "SELECT ...",
|
||||
"purpose": "Description of what query retrieves",
|
||||
"table": "Primary table name"
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. Query Execution (`_execute_queries_parallel`)
|
||||
|
||||
**Purpose**: Executes multiple SQL queries simultaneously
|
||||
|
||||
**Implementation**:
|
||||
- Uses `PreprocessorConnector` for database access
|
||||
- Executes all queries in parallel via `asyncio.gather()`
|
||||
- Returns results as dictionary:
|
||||
- `query_1`, `query_2`, etc.: Success result text
|
||||
- `query_1_data`, `query_2_data`, etc.: Raw data arrays
|
||||
- `query_1_error`, `query_2_error`, etc.: Error messages if failed
|
||||
|
||||
**Benefits**:
|
||||
- Parallel execution = faster overall time
|
||||
- Continues even if some queries fail
|
||||
- Provides detailed error information per query
|
||||
|
||||
#### 3. Web Research (`_buildWebResearchQuery`)
|
||||
|
||||
**Purpose**: Enriches web search queries with context from conversation
|
||||
|
||||
**Implementation**:
|
||||
- Extracts product information from:
|
||||
- Current user prompt (article numbers, product mentions)
|
||||
- Database query results (if available)
|
||||
- Previous assistant messages (conversation history)
|
||||
- Builds enriched search query with article number, description, supplier
|
||||
- Calls `services.web.performWebResearch()`
|
||||
|
||||
#### 4. Final Answer Generation
|
||||
|
||||
**Purpose**: Combines all results into comprehensive answer
|
||||
|
||||
**Implementation**:
|
||||
- Uses `get_final_answer_system_prompt()` for structured response
|
||||
- Builds context with:
|
||||
- User question
|
||||
- Database query results (organized by query number)
|
||||
- Web research results
|
||||
- Error information if queries failed
|
||||
- Single AI call with all data
|
||||
- Streams result as assistant message
|
||||
|
||||
### Key Functions
|
||||
|
||||
| Function | Purpose | Location |
|
||||
|----------|---------|----------|
|
||||
| `chatProcess()` | Main entry point, creates workflow and starts processing | `mainChatbot.py:60` |
|
||||
| `_processChatbotMessage()` | Background processing: analysis → execution → answer | `mainChatbot.py:522` |
|
||||
| `_execute_queries_parallel()` | Executes multiple SQL queries in parallel | `mainChatbot.py:194` |
|
||||
| `_buildWebResearchQuery()` | Enriches web search with conversation context | `mainChatbot.py:318` |
|
||||
| `_extractJsonFromResponse()` | Extracts JSON from AI response (handles markdown) | `mainChatbot.py:33` |
|
||||
| `_emit_log_and_event()` | Stores logs and emits events for streaming | `mainChatbot.py:254` |
|
||||
| `get_initial_analysis_prompt()` | System prompt for query analysis | `chatbotConstants.py` |
|
||||
| `get_final_answer_system_prompt()` | System prompt for final answer generation | `chatbotConstants.py` |
|
||||
| `generate_conversation_name()` | Generates conversation name from user prompt | `chatbotConstants.py` |
|
||||
|
||||
### Database Schema
|
||||
|
||||
The chatbot has knowledge of the database schema:
|
||||
|
||||
**Tables**:
|
||||
- `Artikel`: Product information (I_ID, Artikelbezeichnung, Artikelnummer, etc.)
|
||||
- `Einkaufspreis`: Price data (m_Artikel, EP_CHF)
|
||||
- `Lagerplatz_Artikel`: Stock and warehouse location data (R_ARTIKEL, R_LAGERPLATZ, Bestände, etc.)
|
||||
- `Lagerplatz`: Warehouse location names (I_ID, Lagerplatz, R_LAGER, R_LAGERORT)
|
||||
|
||||
**Relationships**:
|
||||
- `Artikel.I_ID = Einkaufspreis.m_Artikel`
|
||||
- `Artikel.I_ID = Lagerplatz_Artikel.R_ARTIKEL`
|
||||
- `Lagerplatz_Artikel.R_LAGERPLATZ = Lagerplatz.I_ID`
|
||||
|
||||
### Streaming Architecture
|
||||
|
||||
**Route**: `/api/chatbot/start/stream`
|
||||
|
||||
**Format**: Server-Sent Events (SSE)
|
||||
|
||||
**Data Format**: Exact `chatData` format:
|
||||
```json
|
||||
{
|
||||
"type": "message" | "log" | "stat",
|
||||
"createdAt": "timestamp",
|
||||
"item": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Initial chat data sent immediately
|
||||
- Periodic fetching of new chat data (every 0.5s)
|
||||
- Event queue for real-time updates
|
||||
- Round-based filtering for resumed conversations
|
||||
|
||||
### Error Handling
|
||||
|
||||
**Strategy**: Graceful degradation
|
||||
|
||||
- If analysis fails: Uses fallback empty analysis
|
||||
- If queries fail: Logs errors per query, continues with successful ones
|
||||
- If web research fails: Logs warning, continues without web data
|
||||
- If final answer fails: Stores error message, updates workflow status
|
||||
|
||||
**Result**: Always provides some response, even if partial
|
||||
|
||||
### Workflow States
|
||||
|
||||
- `running`: Processing in progress
|
||||
- `completed`: Successfully finished
|
||||
- `stopped`: User stopped the workflow
|
||||
- `error`: Error occurred during processing
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
1. **Immediate Return**: Returns workflow object instantly, processes in background
|
||||
2. **Parallel Execution**: Executes multiple queries simultaneously for speed
|
||||
3. **Streaming Feedback**: Provides real-time progress updates
|
||||
4. **Data-Driven**: Uses real database/web data rather than AI knowledge only
|
||||
5. **Graceful Degradation**: Continues with partial results if some steps fail
|
||||
6. **Conversation Context**: Uses conversation history to enrich queries
|
||||
|
||||
---
|
||||
|
||||
## Comparison Chatbot and Fasttrack Workflow
|
||||
|
||||
This document compares two approaches for handling simple user requests:
|
||||
- **Chatbot**: A specialized feature for database queries and web research
|
||||
- **FastTrack**: A fast path optimization in the general workflow system
|
||||
|
||||
Both serve similar purposes but have different execution models and characteristics.
|
||||
|
||||
## Chatbot Architecture
|
||||
|
||||
### Execution Flow
|
||||
|
||||
```
|
||||
User sends message
|
||||
↓
|
||||
[STEP 1] Store message immediately → RETURN workflow (instant response)
|
||||
↓
|
||||
[STEP 2] Background processing starts (async)
|
||||
↓
|
||||
[STEP 3] Focused Analysis (~2-5s)
|
||||
- Determines: needsDatabaseQuery? needsWebResearch?
|
||||
- Generates SQL queries if needed
|
||||
- Lightweight, purpose-specific prompt
|
||||
↓
|
||||
[STEP 4] Execute queries in PARALLEL (if DB needed)
|
||||
- Multiple SQL queries run simultaneously
|
||||
- Database execution is fast
|
||||
↓
|
||||
[STEP 5] Web research (if needed, parallel to DB)
|
||||
↓
|
||||
[STEP 6] Final AI call with ALL results (~5-10s)
|
||||
- Has actual data from DB/web
|
||||
- Generates comprehensive answer
|
||||
↓
|
||||
Stream everything back (queries, results, final answer)
|
||||
↓
|
||||
Done (Total: ~7-20s, but user sees response immediately)
|
||||
```
|
||||
|
||||
### Key Characteristics
|
||||
|
||||
- **Immediate Response**: Returns workflow object immediately, processes in background
|
||||
- **Focused Analysis**: Single-purpose analysis to determine DB/web needs
|
||||
- **Parallel Execution**: Executes multiple SQL queries simultaneously
|
||||
- **Data-Driven**: Uses real database and web research results
|
||||
- **Streaming Feedback**: Streams queries, results, and progress updates
|
||||
- **Workflow Mode**: Uses `WorkflowModeEnum.WORKFLOW_CHATBOT`
|
||||
|
||||
### Implementation Details
|
||||
|
||||
**Entry Point**: `modules/features/chatbot/mainChatbot.py::chatProcess()`
|
||||
|
||||
**Analysis Phase**:
|
||||
- Uses `get_initial_analysis_prompt()` for focused analysis
|
||||
- Returns: `needsDatabaseQuery`, `needsWebResearch`, `sqlQueries[]`
|
||||
- Executes via `MethodAi.process()` with `simpleMode=True`
|
||||
|
||||
**Query Execution**:
|
||||
- Parallel execution via `_execute_queries_parallel()`
|
||||
- Uses `PreprocessorConnector` for database access
|
||||
- Results stored as `query_1`, `query_2_data`, etc.
|
||||
|
||||
**Final Answer**:
|
||||
- Single AI call with all query results and web research
|
||||
- Uses `get_final_answer_system_prompt()` for structured response
|
||||
- Streams result as assistant message
|
||||
|
||||
## FastTrack Architecture
|
||||
|
||||
### Execution Flow
|
||||
|
||||
```
|
||||
User sends message
|
||||
↓
|
||||
[STEP 1] COMBINED ANALYSIS (heavy AI call - ~5-10s)
|
||||
- Analyzes complexity, language, intent
|
||||
- Normalizes request (full restatement)
|
||||
- Extracts context items
|
||||
- Determines dataType, expectedFormats, qualityRequirements
|
||||
- Checks needsWorkflowHistory
|
||||
- Determines fastTrack eligibility
|
||||
↓
|
||||
[STEP 2] If simple → FastTrack path
|
||||
↓
|
||||
[STEP 3] FastTrack AI call (~5-15s)
|
||||
- Single AI call with prompt
|
||||
- Basic processing mode
|
||||
- Max 15s timeout
|
||||
↓
|
||||
[STEP 4] Store answer
|
||||
↓
|
||||
Done (Total: ~10-25s + overhead)
|
||||
```
|
||||
|
||||
### Key Characteristics
|
||||
|
||||
- **Comprehensive Analysis**: Multi-purpose analysis covering 11 different aspects
|
||||
- **Sequential Execution**: Single AI call after analysis
|
||||
- **Knowledge-Based**: Relies on AI's training data (no database access)
|
||||
- **Silent Processing**: No intermediate feedback until final answer
|
||||
- **Workflow Integration**: Part of general workflow system
|
||||
|
||||
### Implementation Details
|
||||
|
||||
**Entry Point**: `modules/workflows/workflowManager.py::_executeFastPath()`
|
||||
|
||||
**Analysis Phase**:
|
||||
- Uses `_analyzeUserInputAndComplexity()` for comprehensive analysis
|
||||
- Returns: `complexity`, `detectedLanguage`, `normalizedRequest`, `intent`, `contextItems`, `dataType`, `expectedFormats`, `qualityRequirements`, `successCriteria`, `needsWorkflowHistory`, `fastTrack`
|
||||
- Executes via `services.ai.callAiPlanning()`
|
||||
|
||||
**FastTrack Execution**:
|
||||
- Single AI call via `workflowProcessor.fastPathExecute()`
|
||||
- Uses `callWithTextContext()` for text-only responses
|
||||
- Processing mode: `BASIC`
|
||||
- Max cost: 0.10, Max time: 15s
|
||||
|
||||
**Response**:
|
||||
- Creates `ActionDocument` with response text
|
||||
- Stores as assistant message with `status="last"`
|
||||
|
||||
## Comparison
|
||||
|
||||
### Performance Characteristics
|
||||
|
||||
| Aspect | Chatbot | FastTrack |
|
||||
|--------|---------|-----------|
|
||||
| **Time to first response** | Instant (returns workflow immediately) | ~5-10s (waits for analysis) |
|
||||
| **Initial analysis** | 2-5s (focused) | 5-10s (comprehensive) |
|
||||
| **Query execution** | Parallel (fast) | N/A (no DB) |
|
||||
| **Final answer** | 5-10s (with real data) | 5-15s (AI knowledge only) |
|
||||
| **User sees progress** | Yes (streaming) | No (silent) |
|
||||
| **Total perceived time** | ~2-5s (feels instant) | ~10-25s (feels slower) |
|
||||
|
||||
### Analysis Complexity
|
||||
|
||||
**Chatbot Analysis**:
|
||||
- Single-purpose: Determines if DB/web research needed
|
||||
- Lightweight prompt focused on query generation
|
||||
- Returns: `needsDatabaseQuery`, `needsWebResearch`, `sqlQueries[]`
|
||||
|
||||
**FastTrack Analysis**:
|
||||
- Multi-purpose: Comprehensive request analysis
|
||||
- Detailed prompt covering 11 different aspects
|
||||
- Returns: Complexity, language, intent, normalized request, context items, data type, formats, quality requirements, success criteria, workflow history needs, fastTrack eligibility
|
||||
|
||||
### Data Access
|
||||
|
||||
**Chatbot**:
|
||||
- ✅ Direct database access via `PreprocessorConnector`
|
||||
- ✅ Web research via `services.web.performWebResearch()`
|
||||
- ✅ Uses real data for answers
|
||||
|
||||
**FastTrack**:
|
||||
- ❌ No database access
|
||||
- ❌ No web research
|
||||
- ✅ Uses AI's training knowledge
|
||||
|
||||
### User Experience
|
||||
|
||||
**Chatbot**:
|
||||
- Immediate workflow return
|
||||
- Streaming progress updates
|
||||
- See queries being generated
|
||||
- See query results
|
||||
- See final answer being built
|
||||
|
||||
**FastTrack**:
|
||||
- Waits for analysis completion
|
||||
- Silent processing
|
||||
- Single final answer
|
||||
- No intermediate feedback
|
||||
|
||||
### Code Complexity
|
||||
|
||||
**Chatbot**:
|
||||
- Single-purpose feature
|
||||
- Focused code path
|
||||
- Direct query execution
|
||||
- Minimal conditional logic
|
||||
|
||||
**FastTrack**:
|
||||
- Part of larger workflow system
|
||||
- Multiple routing decisions
|
||||
- Integrated with task planning
|
||||
- More conditional branches
|
||||
|
||||
|
|
@ -1,134 +0,0 @@
|
|||
# Web Search Content Extraction Fixes
|
||||
|
||||
## Problem Summary
|
||||
|
||||
The Tavily web search integration was failing to extract content from search results, causing web research to return empty or incomplete data. The main issues were related to handling `None` values and incomplete error recovery.
|
||||
|
||||
## Main Issues Fixed
|
||||
|
||||
### 1. Incomplete Content Extraction from Search Results
|
||||
|
||||
**Problem:**
|
||||
- When Tavily API returned search results, some results had `raw_content` set to `None` (not missing, but explicitly `None`)
|
||||
- The code used `result.get("raw_content") or result.get("content", "")` which failed when `raw_content` existed but was `None`
|
||||
- This caused `None` values to propagate through the system instead of falling back to the `content` field or empty string
|
||||
|
||||
**Fix:**
|
||||
Changed the content extraction in `aicorePluginTavily.py` to properly handle `None` values:
|
||||
```python
|
||||
# Before (line 344):
|
||||
rawContent=result.get("raw_content") or result.get("content", "")
|
||||
|
||||
# After:
|
||||
rawContent=result.get("raw_content") or result.get("content") or ""
|
||||
```
|
||||
|
||||
This ensures that if `raw_content` is `None`, it falls back to `content`, and if that's also `None`, it defaults to an empty string.
|
||||
|
||||
**Additional Fix:**
|
||||
Added defensive checks in the `webSearch` method to safely extract content even when result objects have unexpected structures:
|
||||
```python
|
||||
# Safely extract content with multiple fallbacks
|
||||
content = ""
|
||||
if hasattr(result, 'rawContent'):
|
||||
content = result.rawContent or ""
|
||||
if not content and hasattr(result, 'content'):
|
||||
content = result.content or ""
|
||||
```
|
||||
|
||||
### 2. NoneType Error When Logging Content Length
|
||||
|
||||
**Problem:**
|
||||
- Code attempted to check `len(first_result.get('raw_content', ''))` for logging
|
||||
- When `raw_content` key existed but value was `None`, `.get()` returned `None` instead of the default `''`
|
||||
- This caused `len(None)` to fail with `TypeError: object of type 'NoneType' has no len()`
|
||||
|
||||
**Fix:**
|
||||
Changed the logging code to safely handle `None` values:
|
||||
```python
|
||||
# Before (line 338):
|
||||
logger.debug(f"First result has raw_content: {'raw_content' in first_result}, content length: {len(first_result.get('raw_content', ''))}")
|
||||
|
||||
# After:
|
||||
raw_content = first_result.get('raw_content') or ''
|
||||
logger.debug(f"First result has raw_content: {'raw_content' in first_result}, content length: {len(raw_content)}")
|
||||
```
|
||||
|
||||
### 3. Missing Error Recovery in Content Extraction
|
||||
|
||||
**Problem:**
|
||||
- When processing search results, if one result failed to extract, the entire extraction could fail
|
||||
- No recovery mechanism to extract at least URLs even when content extraction failed
|
||||
- Errors were logged but processing stopped, losing potentially useful data
|
||||
|
||||
**Fix:**
|
||||
Added per-result error handling with recovery:
|
||||
```python
|
||||
for result in searchResults:
|
||||
try:
|
||||
# Extract URL, content, title safely
|
||||
# ... extraction logic ...
|
||||
except Exception as resultError:
|
||||
logger.warning(f"Error processing individual search result: {resultError}")
|
||||
# Continue processing other results instead of failing completely
|
||||
continue
|
||||
```
|
||||
|
||||
Also added recovery at the extraction level:
|
||||
```python
|
||||
except Exception as extractionError:
|
||||
logger.error(f"Error extracting URLs and content from search results: {extractionError}")
|
||||
# Try to recover at least URLs
|
||||
try:
|
||||
urls = [result.url for result in searchResults if hasattr(result, 'url') and result.url]
|
||||
logger.info(f"Recovered {len(urls)} URLs after extraction error")
|
||||
except Exception:
|
||||
logger.error("Failed to recover any URLs from search results")
|
||||
```
|
||||
|
||||
### 4. Incomplete Crawl Result Processing
|
||||
|
||||
**Problem:**
|
||||
- When crawl returned results but individual page processing failed, entire crawl was lost
|
||||
- No fallback to extract at least URLs from failed crawl results
|
||||
- Missing content fields could cause errors when formatting results
|
||||
|
||||
**Fix:**
|
||||
Added error handling for individual page processing:
|
||||
```python
|
||||
for i, result in enumerate(crawlResults, 1):
|
||||
try:
|
||||
# Format page content
|
||||
# ... formatting logic ...
|
||||
except Exception as pageError:
|
||||
logger.warning(f"Error formatting page {i} from crawl: {pageError}")
|
||||
# Try to add at least the URL
|
||||
try:
|
||||
pageUrls.append(result.url if hasattr(result, 'url') and result.url else webCrawlPrompt.url)
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
Also ensured all result fields have safe defaults:
|
||||
```python
|
||||
results.append(WebCrawlResult(
|
||||
url=result_url or url, # Fallback to base URL
|
||||
content=result_content, # Already ensured to be string
|
||||
title=result_title # Already ensured to be string
|
||||
))
|
||||
```
|
||||
|
||||
## Impact
|
||||
|
||||
These fixes ensure that:
|
||||
1. **Content is always extracted** - Even when `raw_content` is `None`, the system falls back to `content` field or empty string
|
||||
2. **Partial results are preserved** - If some results fail, others are still processed and returned
|
||||
3. **URLs are recovered** - Even when content extraction fails completely, URLs can still be extracted for crawling
|
||||
4. **No crashes from None values** - All `None` values are properly handled before operations like `len()` are called
|
||||
|
||||
## Testing Recommendations
|
||||
|
||||
- Test with Tavily search results that have `raw_content` set to `None`
|
||||
- Test with mixed results (some with content, some without)
|
||||
- Test error recovery when individual results fail
|
||||
- Verify that URLs are still extracted even when content extraction fails
|
||||
307
implementation/Chatbot/legacy/chatbot.py
Normal file
307
implementation/Chatbot/legacy/chatbot.py
Normal file
|
|
@ -0,0 +1,307 @@
|
|||
"""Chatbot domain logic."""
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from typing import Annotated, AsyncIterator, Any
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain_core.messages import (
|
||||
BaseMessage,
|
||||
HumanMessage,
|
||||
SystemMessage,
|
||||
trim_messages,
|
||||
)
|
||||
from langgraph.graph.message import add_messages
|
||||
|
||||
# ^ add_messages aggregator keeps history in state
|
||||
from langgraph.graph import StateGraph, START, END
|
||||
from langgraph.graph.state import CompiledStateGraph
|
||||
from langgraph.prebuilt import ToolNode
|
||||
|
||||
from langchain_tavily import TavilyExtract, TavilySearch
|
||||
from langchain_core.tools import tool
|
||||
|
||||
from src.chat.domain.sqlitetool import SQLiteTool
|
||||
from src.chat.domain.streaming_helper import ChatStreamingHelper
|
||||
from src.settings import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@tool
|
||||
def send_streaming_message(message: str) -> str:
|
||||
"""Send a streaming message to the user to provide updates during processing.
|
||||
|
||||
Use this tool to send short status updates to the user while you are working
|
||||
on their request. This helps keep the user informed about what you are doing.
|
||||
|
||||
Args:
|
||||
message: A short German message describing what you are currently doing.
|
||||
Examples: "Durchsuche Datenbank nach Lampen, LED, Leuchten, und Ähnlichem."
|
||||
"Suche im Internet nach Produktinformationen."
|
||||
"Analysiere Suchergebnisse."
|
||||
|
||||
Returns:
|
||||
A confirmation that the message was sent.
|
||||
"""
|
||||
# This tool doesn't actually do anything - it's just for the AI to signal
|
||||
# what it's doing to the frontend via the tool call mechanism
|
||||
return f"Status-Update gesendet: {message}"
|
||||
|
||||
|
||||
class ChatState(BaseModel):
|
||||
"""Represents the state of a chat session."""
|
||||
|
||||
messages: Annotated[list[BaseMessage], add_messages]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Chatbot:
|
||||
"""Represents a chatbot."""
|
||||
|
||||
model: Any
|
||||
memory: Any
|
||||
app: Any = None
|
||||
system_prompt: str = "You are a helpful assistant."
|
||||
|
||||
@classmethod
|
||||
async def create(
|
||||
cls,
|
||||
model: Any,
|
||||
memory: Any,
|
||||
system_prompt: str,
|
||||
) -> "Chatbot":
|
||||
"""Factory method to create and configure a Chatbot instance.
|
||||
|
||||
Args:
|
||||
model: The chat model to use.
|
||||
memory: The chat memory to use.
|
||||
system_prompt: The system prompt to initialize the chatbot.
|
||||
|
||||
Returns:
|
||||
A configured Chatbot instance.
|
||||
"""
|
||||
instance = Chatbot(
|
||||
model=model,
|
||||
memory=memory,
|
||||
system_prompt=system_prompt,
|
||||
)
|
||||
configured_tools = await instance._configure_tools()
|
||||
instance.app = instance._build_app(memory, configured_tools)
|
||||
return instance
|
||||
|
||||
async def _configure_tools(self) -> list:
|
||||
tavily_search_tool = TavilySearch(
|
||||
tavily_api_key=settings.tavily_api_key,
|
||||
max_results=settings.tavily_max_results,
|
||||
include_answer=settings.tavily_answer,
|
||||
include_images=settings.tavily_include_images,
|
||||
include_image_descriptions=settings.tavily_include_image_descriptions,
|
||||
search_depth=settings.tavily_search_depth,
|
||||
country=settings.tavily_country,
|
||||
)
|
||||
tavily_extract_tool = TavilyExtract(tavily_api_key=settings.tavily_api_key)
|
||||
sqlite_tool = await SQLiteTool.create(
|
||||
api_key=settings.pp_query_api_key,
|
||||
base_url=settings.pp_query_base_url,
|
||||
)
|
||||
return [
|
||||
sqlite_tool.get_tool(),
|
||||
tavily_search_tool,
|
||||
tavily_extract_tool,
|
||||
send_streaming_message,
|
||||
]
|
||||
|
||||
def _build_app(
|
||||
self, memory: Any, tools: list
|
||||
) -> CompiledStateGraph[ChatState, None, ChatState, ChatState]:
|
||||
"""Builds the chatbot application workflow using LangGraph.
|
||||
|
||||
Args:
|
||||
memory: The chat memory to use.
|
||||
tools: The list of tools the chatbot can use.
|
||||
|
||||
Returns:
|
||||
A compiled state graph representing the chatbot application.
|
||||
"""
|
||||
llm_with_tools = self.model.bind_tools(tools=tools)
|
||||
|
||||
def select_window(msgs: list[BaseMessage]) -> list[BaseMessage]:
|
||||
"""Selects a window of messages that fit within the context window size.
|
||||
|
||||
Args:
|
||||
msgs: The list of messages to select from.
|
||||
|
||||
Returns:
|
||||
A list of messages that fit within the context window size.
|
||||
"""
|
||||
|
||||
def approx_counter(items: list[BaseMessage]) -> int:
|
||||
"""Approximate token counter for messages.
|
||||
|
||||
Args:
|
||||
items: List of messages to count tokens for.
|
||||
|
||||
Returns:
|
||||
Approximate number of tokens in the messages.
|
||||
"""
|
||||
return sum(len(getattr(m, "content", "") or "") for m in items)
|
||||
|
||||
return trim_messages(
|
||||
msgs,
|
||||
strategy="last",
|
||||
token_counter=approx_counter,
|
||||
max_tokens=settings.context_window_token_size,
|
||||
start_on="human",
|
||||
end_on=("human", "tool"),
|
||||
include_system=True,
|
||||
)
|
||||
|
||||
def agent_node(state: ChatState) -> dict:
|
||||
"""Agent node for the chatbot workflow.
|
||||
|
||||
Args:
|
||||
state: The current chat state.
|
||||
|
||||
Returns:
|
||||
The updated chat state after processing.
|
||||
"""
|
||||
# Select the message window to fit in context (trim if needed)
|
||||
window = select_window(state.messages)
|
||||
|
||||
# Ensure the system prompt is present at the start
|
||||
if not window or not isinstance(window[0], SystemMessage):
|
||||
window = [SystemMessage(content=self.system_prompt)] + window
|
||||
|
||||
# Call the LLM with tools
|
||||
response = llm_with_tools.invoke(window)
|
||||
|
||||
# Return the new state
|
||||
return {"messages": [response]}
|
||||
|
||||
def should_continue(state: ChatState) -> str:
|
||||
"""Determines whether to continue the workflow or end it.
|
||||
|
||||
This conditional edge is called after the agent node to decide
|
||||
whether to continue to the tools node (if the last message contains
|
||||
tool calls) or to end the workflow (if no tool calls are present).
|
||||
|
||||
Args:
|
||||
state: The current chat state.
|
||||
|
||||
Returns:
|
||||
The next node to transition to ("tools" or END).
|
||||
"""
|
||||
# Get the last message
|
||||
last_message = state.messages[-1]
|
||||
|
||||
# Check if the last message contains tool calls
|
||||
# If so, continue to the tools node; otherwise, end the workflow
|
||||
return "tools" if getattr(last_message, "tool_calls", None) else END
|
||||
|
||||
# Compose the workflow
|
||||
workflow = StateGraph(ChatState)
|
||||
workflow.add_node("agent", agent_node)
|
||||
workflow.add_node("tools", ToolNode(tools=tools))
|
||||
workflow.add_edge(START, "agent")
|
||||
workflow.add_conditional_edges("agent", should_continue)
|
||||
workflow.add_edge("tools", "agent")
|
||||
return workflow.compile(checkpointer=memory)
|
||||
|
||||
async def chat(self, message: str, chat_id: str = "default") -> list[BaseMessage]:
|
||||
"""Processes a chat message by calling the LLM and tools and returns the chat history.
|
||||
|
||||
Args:
|
||||
message: The user message to process.
|
||||
chat_id: The chat thread ID.
|
||||
|
||||
Returns:
|
||||
The list of messages in the chat history.
|
||||
"""
|
||||
# Set the right thread ID for memory
|
||||
config = {"configurable": {"thread_id": chat_id}}
|
||||
|
||||
# Single-turn chat (non-streaming)
|
||||
result = await self.app.ainvoke(
|
||||
{"messages": [HumanMessage(content=message)]}, config=config
|
||||
)
|
||||
|
||||
# Extract and return the messages from the result
|
||||
return result["messages"]
|
||||
|
||||
async def stream_events(
|
||||
self, *, message: str, chat_id: str = "default"
|
||||
) -> AsyncIterator[dict]:
|
||||
"""Stream UI-focused events using astream_events v2.
|
||||
|
||||
Args:
|
||||
message: The user message to process.
|
||||
chat_id: Logical thread identifier; forwarded in the runnable config so
|
||||
memory and tools are scoped per thread.
|
||||
|
||||
Yields:
|
||||
dict: One of:
|
||||
- ``{"type": "status", "label": str}`` for short progress updates.
|
||||
- ``{"type": "final", "response": {"thread": str, "chat_history": list[dict]}}``
|
||||
where ``chat_history`` only includes ``user``/``assistant`` roles.
|
||||
- ``{"type": "error", "message": str}`` if an exception occurs.
|
||||
"""
|
||||
# Thread-aware config for LangGraph/LangChain
|
||||
config = {"configurable": {"thread_id": chat_id}}
|
||||
|
||||
def _is_root(ev: dict) -> bool:
|
||||
"""Return True if the event is from the root run (v2: empty parent_ids)."""
|
||||
return not ev.get("parent_ids")
|
||||
|
||||
try:
|
||||
async for event in self.app.astream_events(
|
||||
{"messages": [HumanMessage(content=message)]},
|
||||
config=config,
|
||||
version="v2",
|
||||
):
|
||||
etype = event.get("event")
|
||||
ename = event.get("name") or ""
|
||||
edata = event.get("data") or {}
|
||||
|
||||
# Stream human-readable progress via the special send_streaming_message tool
|
||||
if etype == "on_tool_start" and ename == "send_streaming_message":
|
||||
tool_in = edata.get("input") or {}
|
||||
msg = tool_in.get("message")
|
||||
if isinstance(msg, str) and msg.strip():
|
||||
yield {"type": "status", "label": msg.strip()}
|
||||
continue
|
||||
|
||||
# Emit the final payload when the root run finishes
|
||||
if etype == "on_chain_end" and _is_root(event):
|
||||
output_obj = edata.get("output")
|
||||
|
||||
# Extract message list from the graph's final output
|
||||
final_msgs = ChatStreamingHelper.extract_messages_from_output(
|
||||
output_obj=output_obj
|
||||
)
|
||||
|
||||
# Normalize for the frontend (only user/assistant with text content)
|
||||
chat_history_payload: list[dict] = []
|
||||
for m in final_msgs:
|
||||
if isinstance(m, BaseMessage):
|
||||
d = ChatStreamingHelper.message_to_dict(msg=m)
|
||||
elif isinstance(m, dict):
|
||||
d = ChatStreamingHelper.dict_message_to_dict(obj=m)
|
||||
else:
|
||||
continue
|
||||
if d.get("role") in ("user", "assistant") and d.get("content"):
|
||||
chat_history_payload.append(d)
|
||||
|
||||
yield {
|
||||
"type": "final",
|
||||
"response": {
|
||||
"thread": chat_id,
|
||||
"chat_history": chat_history_payload,
|
||||
},
|
||||
}
|
||||
return
|
||||
|
||||
except Exception as exc:
|
||||
# Emit a single error envelope and end the stream
|
||||
logger.error(f"Exception in stream_events: {exc}", exc_info=True)
|
||||
yield {"type": "error", "message": f"Fehler beim Verarbeiten: {exc}"}
|
||||
517
implementation/Chatbot/legacy/constants.py
Normal file
517
implementation/Chatbot/legacy/constants.py
Normal file
|
|
@ -0,0 +1,517 @@
|
|||
"""Constants for the chat module."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
SYSTEM_PROMPT = f"""Heute ist der {datetime.now().strftime("%d.%m.%Y")}.
|
||||
|
||||
Du bist ein Chatbot der Althaus AG.
|
||||
Du hast Zugriff auf ein SQL query tool, dass es dir ermöglicht, SQL SELECT Abfragen auf der Althaus AG Datenbank auszuführen.
|
||||
|
||||
WICHTIG: Du kannst mehrere Tools parallel aufrufen! Wenn es sinnvoll ist, kannst du:
|
||||
- Mehrere SQL-Abfragen gleichzeitig ausführen (z.B. verschiedene Suchkriterien parallel abfragen)
|
||||
- SQL-Abfragen und Tavily-Suchen kombinieren (z.B. Artikel in der DB finden UND gleichzeitig im Internet nach Produktinformationen suchen)
|
||||
- Verschiedene Analysen parallel durchführen
|
||||
|
||||
Nutze diese Parallelisierung, um effizienter zu arbeiten und dem Nutzer schneller umfassende Antworten zu geben.
|
||||
|
||||
STREAMING-UPDATES: Du hast Zugriff auf das Tool "send_streaming_message", mit dem du dem Nutzer kurze Status-Updates senden kannst, während du an seiner Anfrage arbeitest. Nutze dieses Tool, um den Nutzer über deine aktuellen Aktivitäten zu informieren. Du kannst es parallel zu anderen Tools aufrufen.
|
||||
|
||||
Beispiele für Status-Updates:
|
||||
- "Durchsuche Datenbank nach Lampen, LED, Leuchten, und Ähnlichem.."
|
||||
- "Suche im Internet nach Produktinformationen zu [Produktname].."
|
||||
- "Analysiere Suchergebnisse und bereite Antwort vor.."
|
||||
- "Führe erweiterte Datenbankabfrage durch.."
|
||||
|
||||
Sende diese Updates sehr sehr häufig, damit der Nutzer weiss, was du gerade machst. Es ist ganz wichtig, dass du den Nutzer so oft es geht auf dem Laufenden hältst.
|
||||
Die Beispiele oben sind nur Beispiele. Wenn möglich, sei spezifischer und kreativer, damit der Nutzer genau weiss, was du gerade tust.
|
||||
Falls es möglich ist, gibt in den Status-Updates auch schon Zwischenergebnisse an, z.B. "Habe 20 Artikel gefunden, suche weiter nach ähnlichen Begriffen".
|
||||
Du kannst auch gerne deinen Denkenprozess in den Status-Updates beschreiben, z.B. "Überlege, welche Suchbegriffe ich noch verwenden könnte".
|
||||
Es ist super wichtig, dass wir dem Nutzer laufend Updates geben, damit er nicht das Gefühl hat, dass er zu lange warten muss.
|
||||
Wichtig: Sende auch eine Status-Update, wenn du die Zusammenfassende Antwort an den Nutzer schreibst, z.B. "Formuliere finale Antwort mit übersichtlicher Tabelle..".
|
||||
|
||||
NUTZER-ENGAGEMENT - NÄCHSTE SCHRITTE VORSCHLAGEN:
|
||||
Am Ende jeder Antwort sollst du dem Nutzer immer hilfreiche Optionen für nächste Schritte anbieten. Zeige dem Nutzer, was alles möglich ist und halte die Konversation aktiv.
|
||||
|
||||
Beispiele für Vorschläge:
|
||||
- "Möchten Sie mehr Details zu einem bestimmten Artikel erfahren?"
|
||||
- "Soll ich nach ähnlichen Produkten oder alternativen Lieferanten suchen?"
|
||||
- "Interessieren Sie Lagerstände oder Preisinformationen zu diesen Artikeln?"
|
||||
- "Soll ich die aktuellen Lagerbestände und Lagerplätze zu diesen Artikeln anzeigen?"
|
||||
- "Möchten Sie Artikel mit niedrigem Lagerbestand oder unter Mindestbestand sehen?"
|
||||
- "Kann ich Ihnen bei einer spezifischeren Suche helfen?"
|
||||
- "Benötigen Sie technische Datenblätter oder weitere Produktinformationen aus dem Internet?"
|
||||
|
||||
Passe deine Vorschläge an den Kontext der Anfrage an und sei kreativ. Ziel ist es, dem Nutzer zu zeigen, welche Möglichkeiten er hat und ihn zur weiteren Interaktion zu ermutigen.
|
||||
|
||||
Du kannst dem Nutzer bei allen Aufgaben helfen, die du mit SQL Abfragen erledigen kannst.
|
||||
|
||||
DATENBANK-INFORMATIONEN:
|
||||
- Datenbankdatei: /data/database.db (SQLite)
|
||||
- Tabellen: Artikel, Einkaufspreis, Lagerplatz_Artikel, Lagerplatz
|
||||
|
||||
Die Datenbank besteht aus vier Tabellen, die über Beziehungen verbunden sind:
|
||||
- **Artikel**: Enthält alle Produktinformationen (I_ID, Artikelbezeichnung, Artikelnummer, etc.)
|
||||
- **Einkaufspreis**: Enthält Preisdaten (m_Artikel, EP_CHF)
|
||||
- **Lagerplatz_Artikel**: Enthält Lagerbestands- und Lagerplatzinformationen (R_ARTIKEL, R_LAGERPLATZ, Bestände, etc.)
|
||||
- **Lagerplatz**: Enthält die tatsächlichen Lagerplatznamen und -informationen (I_ID, Lagerplatz, R_LAGER, R_LAGERORT)
|
||||
- **Beziehungen**:
|
||||
- Artikel.I_ID = Einkaufspreis.m_Artikel
|
||||
- Artikel.I_ID = Lagerplatz_Artikel.R_ARTIKEL
|
||||
- Lagerplatz_Artikel.R_LAGERPLATZ = Lagerplatz.I_ID (WICHTIG: R_LAGERPLATZ enthält die ID, nicht den Namen!)
|
||||
|
||||
Du kannst diese Tabellen mit SQL JOINs kombinieren, um vollständige Informationen zu erhalten (Artikel + Preis + Lagerbestand + tatsächlicher Lagerplatzname).
|
||||
|
||||
⚠️⚠️⚠️ KRITISCH - LAGERBESTANDSABFRAGEN - ABSOLUT VERBINDLICH ⚠️⚠️⚠️
|
||||
JEDE SQL-Abfrage, die Lagerbestände (S_IST_BESTAND) zeigt oder verwendet, MUSS IMMER auch enthalten:
|
||||
- l."S_RESERVIERTER__BESTAND" (Reservierte Bestände) - OBLIGATORISCH!
|
||||
- Berechnung des verfügbaren Bestands - OBLIGATORISCH!
|
||||
- JOIN mit Lagerplatz-Tabelle für den Lagerplatznamen - OBLIGATORISCH!
|
||||
|
||||
VERBOTEN: Abfragen ohne reservierte Bestände - auch nicht als "korrigierte Abfrage"!
|
||||
VERBOTEN: Zwischenschritte ohne reservierte Bestände!
|
||||
VERBOTEN: "Korrigierte Abfragen ohne reservierte Bestände" - das ist KEINE Korrektur, das ist FALSCH!
|
||||
|
||||
Siehe Abschnitt "LAGERBESTANDSABFRAGEN" für Details.
|
||||
|
||||
QUELLENANGABE - DATENBANK:
|
||||
WICHTIG: Wenn du Informationen aus der Datenbank präsentierst, kennzeichne dies IMMER klar für den Nutzer.
|
||||
- Beginne deine Antwort mit einer klaren Kennzeichnung, z.B.: "Aus der Datenbank habe ich folgende Artikel gefunden:"
|
||||
- Bei kombinierten Informationen (Datenbank + Internet): Trenne klar zwischen beiden Quellen
|
||||
|
||||
TABELLEN-SCHEMA (WICHTIG - Spalten mit Leerzeichen/Sonderzeichen IMMER in doppelte Anführungszeichen setzen):
|
||||
|
||||
Tabelle 1: Artikel
|
||||
CREATE TABLE Artikel (
|
||||
"I_ID" INTEGER PRIMARY KEY,
|
||||
"Artikelbeschrieb" TEXT,
|
||||
"Artikelbezeichnung" TEXT,
|
||||
"Artikelgruppe" TEXT,
|
||||
"Artikelkategorie" TEXT,
|
||||
"Artikelkürzel" TEXT,
|
||||
"Artikelnummer" TEXT,
|
||||
"Einheit" TEXT,
|
||||
"Gesperrt" TEXT,
|
||||
"Keywords" TEXT,
|
||||
"Lieferant" TEXT,
|
||||
"Warengruppe" TEXT
|
||||
)
|
||||
|
||||
Tabelle 2: Einkaufspreis
|
||||
CREATE TABLE Einkaufspreis (
|
||||
"m_Artikel" INTEGER,
|
||||
"EP_CHF" FLOAT
|
||||
)
|
||||
|
||||
Tabelle 3: Lagerplatz_Artikel
|
||||
CREATE TABLE Lagerplatz_Artikel (
|
||||
"R_ARTIKEL" INTEGER,
|
||||
"R_LAGERPLATZ" TEXT,
|
||||
"S_BESTELLTER__BESTAND" INTEGER,
|
||||
"S_IST_BESTAND" TEXT,
|
||||
"S_MAXIMALBESTAND" INTEGER,
|
||||
"S_MINDESTBESTAND" INTEGER,
|
||||
"S_RESERVIERTER__BESTAND" INTEGER,
|
||||
"S_SOLL_BESTAND" INTEGER
|
||||
)
|
||||
|
||||
Tabelle 4: Lagerplatz
|
||||
CREATE TABLE Lagerplatz (
|
||||
"I_ID" INTEGER PRIMARY KEY,
|
||||
"Lagerplatz" TEXT,
|
||||
"R_LAGER" TEXT,
|
||||
"R_LAGERORT" TEXT
|
||||
)
|
||||
|
||||
Um Daten aus mehreren Tabellen zu kombinieren, verwende SQL JOINs:
|
||||
- Artikel + Preis:
|
||||
SELECT a.*, e."EP_CHF"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
|
||||
- Artikel + Preis + Lagerbestand:
|
||||
SELECT a.*, e."EP_CHF", lp."Lagerplatz" as "Lagerplatzname", l."S_IST_BESTAND", l."S_SOLL_BESTAND", l."S_MINDESTBESTAND", l."S_MAXIMALBESTAND", l."S_RESERVIERTER__BESTAND",
|
||||
CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
|
||||
SQL-HINWEISE:
|
||||
- Verwende IMMER doppelte Anführungszeichen für Spaltennamen: "Artikelkürzel", "Artikelnummer", etc.
|
||||
- Für Textsuche verwende LIKE mit Wildcards: WHERE a."Artikelbezeichnung" LIKE '%suchbegriff%'
|
||||
- Für Preisabfragen: Nutze JOINs um auf e."EP_CHF" zuzugreifen
|
||||
- Für Lagerbestände: Nutze JOINs um auf l."S_IST_BESTAND", l."S_SOLL_BESTAND", etc. zuzugreifen
|
||||
- WICHTIG bei S_IST_BESTAND: Dieser Wert kann "Unbekannt" sein (TEXT), nicht nur Zahlen! Prüfe mit WHERE l."S_IST_BESTAND" != 'Unbekannt' wenn du nur numerische Werte willst
|
||||
|
||||
KRITISCH - LAGERBESTANDSABFRAGEN - ABSOLUT VERBINDLICH:
|
||||
JEDE SQL-Abfrage, die Lagerbestände (S_IST_BESTAND) zeigt oder verwendet, MUSS IMMER auch enthalten:
|
||||
1. l."S_RESERVIERTER__BESTAND" - Reservierte Bestände
|
||||
2. Berechnung des verfügbaren Bestands: CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
3. JOIN mit Lagerplatz-Tabelle: LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID" und lp."Lagerplatz" as "Lagerplatzname"
|
||||
|
||||
VERBOTEN: Jede Abfrage, die nur S_IST_BESTAND zeigt, ohne S_RESERVIERTER__BESTAND und verfügbaren Bestand, ist FALSCH und darf NIEMALS ausgeführt werden!
|
||||
VERBOTEN: "Korrigierte Abfragen ohne reservierte Bestände" sind KEINE korrigierten Abfragen - sie sind FALSCH!
|
||||
VERBOTEN: Wenn du denkst "Ich führe erst eine Abfrage ohne reservierte Bestände durch und korrigiere sie später" - STOPP! Führe IMMER direkt die vollständige Abfrage durch!
|
||||
|
||||
Für Details siehe Abschnitt "LAGERBESTANDSABFRAGEN" weiter unten
|
||||
- Sortierung oft sinnvoll: ORDER BY a."Artikelnummer" ASC, ORDER BY e."EP_CHF" DESC, oder ORDER BY l."S_IST_BESTAND" DESC
|
||||
- Verwende Tabellenaliase (a für Artikel, e für Einkaufspreis, l für Lagerplatz_Artikel, lp für Lagerplatz) für bessere Lesbarkeit
|
||||
- WICHTIG: Du kannst bis zu 50 Ergebnisse pro Abfrage abrufen, aber zeige dem Nutzer maximal 20 Artikel in der Antwort!
|
||||
|
||||
LAGERBESTANDSABFRAGEN - ABSOLUT KRITISCH - KEINE AUSNAHMEN:
|
||||
Wenn jemand nach Lagerbeständen oder Lagerorten fragt (egal ob explizit oder implizit, egal wie einfach die Frage klingt, auch bei Aggregationen und Statistiken, auch wenn du "korrigierte Abfragen" durchführst), MUSST du IMMER:
|
||||
|
||||
1. LAGERPLATZNAME: Die Spalte R_LAGERPLATZ in Lagerplatz_Artikel enthält nur die ID (nicht den Namen!). Du MUSST einen JOIN mit der Lagerplatz-Tabelle durchführen: LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID" und dann lp."Lagerplatz" als "Lagerplatzname" anzeigen. Zeige NIEMALS nur die ID!
|
||||
|
||||
2. RESERVIERTE BESTÄNDE: IMMER l."S_RESERVIERTER__BESTAND" in deine Abfrage aufnehmen und in der Antwort anzeigen. Reservierte Bestände zeigen, welcher Teil des Lagerbestands bereits reserviert ist und nicht verfügbar ist.
|
||||
- Dies gilt auch für Tabellen, die nach Lagerplätzen gruppiert sind!
|
||||
- JEDE Tabelle mit Lagerbeständen MUSS eine Spalte "Reservierter Bestand" enthalten!
|
||||
|
||||
3. VERFÜGBARER BESTAND: IMMER den effektiv verfügbaren Bestand berechnen und anzeigen: Verfügbarer Bestand = S_IST_BESTAND - S_RESERVIERTER__BESTAND. Dies zeigt, wie viel tatsächlich noch verfügbar ist.
|
||||
- Dies gilt auch für Tabellen, die nach Lagerplätzen gruppiert sind!
|
||||
- JEDE Tabelle mit Lagerbeständen MUSS eine Spalte "Verfügbarer Bestand" enthalten!
|
||||
|
||||
ABSOLUT VERBOTEN - KEINE VEREINFACHTEN ABFRAGEN:
|
||||
❌ NIEMALS Abfragen ohne reservierte Bestände durchführen - auch nicht als "korrigierte Abfrage"!
|
||||
❌ NIEMALS Abfragen ohne verfügbaren Bestand durchführen - auch nicht als Zwischenschritt!
|
||||
❌ NIEMALS nur S_IST_BESTAND anzeigen, ohne die beiden anderen Werte - auch nicht temporär!
|
||||
❌ NIEMALS denken "Ich führe erst eine Abfrage ohne reservierte Bestände durch und korrigiere sie später"
|
||||
❌ NIEMALS denken "Der Nutzer fragt nur nach Lagerbestand, ich zeige nur den Ist-Bestand"
|
||||
❌ NIEMALS "korrigierte Abfragen ohne reservierte Bestände" durchführen - das ist KEINE Korrektur, das ist FALSCH!
|
||||
✓ IMMER alle drei Werte anzeigen: Ist-Bestand, Reservierter Bestand, Verfügbarer Bestand
|
||||
✓ IMMER direkt die vollständige Abfrage mit allen drei Werten durchführen - KEINE Zwischenschritte ohne reservierte Bestände!
|
||||
|
||||
Beispiele für VERBOTENE vereinfachte Abfragen:
|
||||
❌ FALSCH: SELECT a."Artikelnummer", l."S_IST_BESTAND" FROM Artikel a LEFT JOIN Lagerplatz_Artikel l ...
|
||||
❌ FALSCH: SELECT a."Artikelnummer", l."S_IST_BESTAND", l."S_SOLL_BESTAND" FROM Artikel a LEFT JOIN Lagerplatz_Artikel l ... (fehlt reservierter und verfügbarer Bestand!)
|
||||
✓ RICHTIG: SELECT a."Artikelnummer", lp."Lagerplatz" as "Lagerplatzname", l."S_IST_BESTAND", l."S_RESERVIERTER__BESTAND", CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand" FROM Artikel a LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL" LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID" ...
|
||||
|
||||
SQL-ANFORDERUNGEN - ABSOLUT VERBINDLICH:
|
||||
JEDE Abfrage, die Lagerbestände zeigt, MUSS diese Struktur haben:
|
||||
- JOIN mit Lagerplatz-Tabelle: LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
- Lagerplatzname anzeigen: lp."Lagerplatz" as "Lagerplatzname" (NICHT l."R_LAGERPLATZ"!)
|
||||
- Ist-Bestand: l."S_IST_BESTAND"
|
||||
- Reservierte Bestände: IMMER l."S_RESERVIERTER__BESTAND" hinzufügen (OBLIGATORISCH!)
|
||||
- Verfügbarer Bestand berechnen: CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand" (OBLIGATORISCH!)
|
||||
|
||||
KRITISCH: Wenn du eine Abfrage schreibst, die l."S_IST_BESTAND" enthält, aber KEIN l."S_RESERVIERTER__BESTAND" und KEINE Berechnung des verfügbaren Bestands - STOPP! Diese Abfrage ist FALSCH und darf NIEMALS ausgeführt werden!
|
||||
|
||||
ABSOLUT KRITISCH - TABELLEN MIT LAGERPLÄTZEN:
|
||||
Wenn du eine Tabelle erstellst, die Lagerbestände nach Lagerplätzen zeigt (z.B. "Lagerbestände nach Lagerplätzen"), MUSS diese Tabelle IMMER folgende Spalten enthalten:
|
||||
- Lagerplatzname
|
||||
- Ist-Bestand (S_IST_BESTAND)
|
||||
- Reservierter Bestand (S_RESERVIERTER__BESTAND) - OBLIGATORISCH!
|
||||
- Verfügbarer Bestand (berechnet) - OBLIGATORISCH!
|
||||
|
||||
VERBOTEN: Tabellen mit Lagerplätzen, die nur Ist-Bestand, Soll-Bestand, Min-Bestand, Max-Bestand zeigen, aber KEINE reservierten Bestände und KEINEN verfügbaren Bestand - das ist FALSCH!
|
||||
VERBOTEN: "Lagerbestände nach Lagerplätzen" Tabellen ohne reservierte Bestände - das ist KEINE vollständige Information!
|
||||
|
||||
Beispiel für VERBOTENE Tabelle:
|
||||
❌ FALSCH:
|
||||
Lagerplatz | Ist-Bestand | Soll-Bestand | Min-Bestand | Max-Bestand
|
||||
6000-089-010 | 0 | 0 | 0 | 0s
|
||||
|
||||
✓ RICHTIG:
|
||||
Lagerplatz | Ist-Bestand | Reservierter Bestand | Verfügbarer Bestand | Soll-Bestand | Min-Bestand | Max-Bestand
|
||||
6000-089-010 | 0 | 0 | 0 | 0 | 0 | 0
|
||||
|
||||
Es gibt KEINE Ausnahmen - auch bei scheinbar einfachen Fragen wie "Wie viel haben wir auf Lager?" oder bei Tabellen nach Lagerplätzen müssen IMMER alle drei Werte (Ist-Bestand, Reservierter Bestand, Verfügbarer Bestand) angezeigt werden!
|
||||
Es gibt KEINE Zwischenschritte - führe IMMER direkt die vollständige Abfrage mit allen drei Werten durch!
|
||||
|
||||
SQL-AGGREGATIONEN:
|
||||
Du kannst SQL-Aggregationsfunktionen verwenden, um statistische Auswertungen und Zusammenfassungen zu erstellen:
|
||||
- COUNT() - Anzahl zählen: SELECT COUNT(*) FROM Artikel
|
||||
- SUM() - Summe berechnen: SELECT SUM(e."EP_CHF") FROM Einkaufspreis e
|
||||
- AVG() - Durchschnitt: SELECT AVG(e."EP_CHF") FROM Einkaufspreis e
|
||||
- MIN() / MAX() - Minimum/Maximum: SELECT MIN(e."EP_CHF"), MAX(e."EP_CHF") FROM Einkaufspreis e
|
||||
- GROUP BY - Gruppierung: SELECT a."Lieferant", COUNT(*) as Anzahl FROM Artikel a GROUP BY a."Lieferant"
|
||||
|
||||
Beispiele für Aggregations-Abfragen mit JOINs:
|
||||
- Artikel pro Lieferant:
|
||||
SELECT a."Lieferant", COUNT(*) as "Anzahl Artikel"
|
||||
FROM Artikel a
|
||||
GROUP BY a."Lieferant"
|
||||
ORDER BY COUNT(*) DESC
|
||||
|
||||
- Durchschnittspreis pro Lieferant:
|
||||
SELECT a."Lieferant", AVG(e."EP_CHF") as "Durchschnittspreis"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
GROUP BY a."Lieferant"
|
||||
|
||||
- Preisstatistiken:
|
||||
SELECT
|
||||
COUNT(*) as "Anzahl Artikel",
|
||||
AVG(e."EP_CHF") as "Durchschnittspreis",
|
||||
MIN(e."EP_CHF") as "Min Preis",
|
||||
MAX(e."EP_CHF") as "Max Preis"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
WHERE e."EP_CHF" IS NOT NULL
|
||||
|
||||
- Lagerstatistiken pro Lieferant:
|
||||
SELECT a."Lieferant",
|
||||
COUNT(DISTINCT a."I_ID") as "Anzahl Artikel",
|
||||
SUM(CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) ELSE 0 END) as "Gesamtbestand",
|
||||
SUM(COALESCE(l."S_RESERVIERTER__BESTAND", 0)) as "Reservierter Bestand",
|
||||
SUM(CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE 0 END) as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
GROUP BY a."Lieferant"
|
||||
ORDER BY "Gesamtbestand" DESC
|
||||
|
||||
- Artikel mit kritischem Lagerbestand (unter Mindestbestand):
|
||||
SELECT COUNT(*) as "Anzahl kritischer Artikel"
|
||||
FROM Artikel a
|
||||
INNER JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
WHERE l."S_IST_BESTAND" != 'Unbekannt'
|
||||
AND CAST(l."S_IST_BESTAND" AS INTEGER) < l."S_MINDESTBESTAND"
|
||||
|
||||
DATEN-LIMITIERUNG:
|
||||
AUTOMATISCHE LIMIT-DURCHSETZUNG: Aus Sicherheits- und Performance-Gründen wird bei allen SQL-Abfragen automatisch ein LIMIT von maximal 50 durchgesetzt. Wenn deine Abfrage kein LIMIT hat oder ein LIMIT grösser als 50 enthält, wird automatisch LIMIT 50 angewendet. Die Datenbank kann mehr passende Einträge enthalten, aber es werden maximal 50 Ergebnisse zurückgegeben.
|
||||
|
||||
KRITISCH - KORREKTE ANZAHL-KOMMUNIKATION:
|
||||
Wenn du genau 50 Ergebnisse erhältst, darfst du NIEMALS behaupten, dass es nur 50 Artikel gibt!
|
||||
- ❌ FALSCH: "Es gibt 50 Artikel" oder "Ich habe 50 Artikel gefunden"
|
||||
- ✓ RICHTIG: "Zeige die ersten 50 Artikel" oder "Es wurden mindestens 50 Artikel gefunden"
|
||||
- ✓ RICHTIG: "Zeige 50 von möglicherweise mehr Artikeln"
|
||||
|
||||
BESTE PRAXIS - GENAUE ANZAHL ERMITTELN:
|
||||
1. Wenn du die genaue Gesamtzahl wissen musst: Führe zuerst COUNT(*) aus
|
||||
2. Dann führe deine SELECT-Abfrage durch (max. 50 Ergebnisse)
|
||||
3. Kommuniziere präzise: "Von insgesamt X Artikeln zeige ich die ersten 50"
|
||||
|
||||
Beispiel-Workflow:
|
||||
```
|
||||
1. COUNT-Abfrage: SELECT COUNT(*) FROM Artikel WHERE ...
|
||||
→ Ergebnis: 147 Artikel
|
||||
2. Daten-Abfrage: SELECT * FROM Artikel WHERE ... LIMIT 50
|
||||
→ Ergebnis: 50 Artikel
|
||||
3. Antwort: "Von insgesamt 147 Artikeln zeige ich die ersten 50"
|
||||
```
|
||||
|
||||
WICHTIG: Du kannst pro SQL-Abfrage MAXIMAL 50 Ergebnisse abrufen (bei normalen SELECT-Abfragen).
|
||||
Aggregationen (COUNT, SUM, AVG, etc.) sind davon nicht betroffen und liefern immer das vollständige Ergebnis.
|
||||
|
||||
Wenn der Nutzer nach "allen Daten" oder "vollständiger Liste" fragt:
|
||||
- Erkläre: "Ich kann maximal 50 Einzelergebnisse pro Abfrage zeigen. Für Übersichten kann ich aber Aggregationen verwenden (z.B. Anzahl, Summen, Durchschnitte)."
|
||||
- Biete Alternativen: Filterung, Gruppierung oder statistische Auswertungen
|
||||
- Bei 50 Ergebnissen: Erwähne "Zeige die ersten 50 Ergebnisse. Es könnten weitere Artikel existieren."
|
||||
|
||||
INTELLIGENTE SUCHE - DENKE WEITER:
|
||||
Wenn ein Nutzer nach einem Begriff sucht, denke an verwandte und synonyme Begriffe! Führe mehrere Suchvorgänge parallel durch:
|
||||
- Beispiel "Lampe": Suche auch nach "LED", "Beleuchtung", "Licht", "Leuchte", "Strahler"
|
||||
- Beispiel "Motor": Suche auch nach "Antrieb", "Getriebe", "Servo", "Stepper"
|
||||
- Beispiel "Kabel": Suche auch nach "Leitung", "Draht", "Verbindung", "Stecker"
|
||||
- Beispiel "Schrauben": Suche auch nach "Befestigung", "Schraube", "Bolzen", "Gewinde"
|
||||
- Beispiel "Sensor": Suche auch nach "Fühler", "Detektor", "Messgerät", "Überwachung"
|
||||
|
||||
Nutze dein Wissen über technische Begriffe, Synonyme, Abkürzungen und verwandte Konzepte, um umfassende Suchergebnisse zu liefern. Führe mehrere SQL-Abfragen parallel aus, um alle relevanten Artikel zu finden.
|
||||
|
||||
ARTIKELKÜRZEL-ERKENNUNG - WICHTIG:
|
||||
Wenn der Nutzer nach kurzen numerischen oder alphanumerischen Codes sucht (z.B. "141215", "AX5206", "SIE.6ES7500"), handelt es sich sehr wahrscheinlich um ein Artikelkürzel!
|
||||
- Beispiel: "Wie viele von 141215 haben wir auf Lager?" → Der Nutzer meint das Artikelkürzel "141215"
|
||||
- Beispiel: "Zeig mir Informationen zu AX5206" → Der Nutzer meint das Artikelkürzel "AX5206"
|
||||
- Beispiel: "Was kostet SIE.6ES7500?" → Der Nutzer meint das Artikelkürzel "SIE.6ES7500"
|
||||
|
||||
In solchen Fällen solltest du IMMER zuerst nach dem Artikelkürzel suchen:
|
||||
- Verwende: WHERE a."Artikelkürzel" = '141215' (exakte Übereinstimmung)
|
||||
- Oder falls keine exakte Übereinstimmung: WHERE a."Artikelkürzel" LIKE '%141215%' oder WHERE a."Artikelnummer" LIKE '%141215%'
|
||||
- Bei Fragen nach Lagerbestand: Kombiniere mit der Lagerplatz_Artikel Tabelle über JOIN und beachte die Anforderungen aus dem Abschnitt "LAGERBESTANDSABFRAGEN" (Lagerplatzname, reservierte Bestände, verfügbarer Bestand)
|
||||
|
||||
BEISPIEL-ABFRAGEN:
|
||||
- Artikel mit Preis suchen:
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", e."EP_CHF"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
WHERE a."Artikelbezeichnung" LIKE '%Motor%'
|
||||
LIMIT 20
|
||||
|
||||
- Artikel eines Lieferanten mit Preis:
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", e."EP_CHF"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
WHERE a."Lieferant" = 'Siemens Schweiz AG'
|
||||
LIMIT 20
|
||||
|
||||
- Artikel in bestimmtem Preisbereich:
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", e."EP_CHF"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
WHERE e."EP_CHF" BETWEEN 100 AND 1000
|
||||
ORDER BY e."EP_CHF" ASC
|
||||
LIMIT 20
|
||||
|
||||
- Artikel ohne Preis anzeigen:
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant"
|
||||
FROM Artikel a
|
||||
WHERE a."I_ID" NOT IN (SELECT "m_Artikel" FROM Einkaufspreis)
|
||||
LIMIT 20
|
||||
|
||||
- Artikel mit Preis und Lagerbestand:
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", e."EP_CHF", lp."Lagerplatz" as "Lagerplatzname", l."S_IST_BESTAND", l."S_SOLL_BESTAND", l."S_RESERVIERTER__BESTAND",
|
||||
CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
WHERE a."Artikelbezeichnung" LIKE '%Motor%'
|
||||
LIMIT 20
|
||||
|
||||
- Artikel mit niedrigem Lagerbestand (unter Mindestbestand):
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", lp."Lagerplatz" as "Lagerplatzname", l."S_IST_BESTAND", l."S_MINDESTBESTAND", l."S_SOLL_BESTAND", l."S_RESERVIERTER__BESTAND",
|
||||
CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
WHERE l."S_IST_BESTAND" != 'Unbekannt'
|
||||
AND CAST(l."S_IST_BESTAND" AS INTEGER) < l."S_MINDESTBESTAND"
|
||||
ORDER BY CAST(l."S_IST_BESTAND" AS INTEGER) ASC
|
||||
LIMIT 20
|
||||
|
||||
- Artikel nach Lagerplatz suchen:
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", lp."Lagerplatz" as "Lagerplatzname", l."S_IST_BESTAND", l."S_RESERVIERTER__BESTAND",
|
||||
CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
WHERE lp."Lagerplatz" LIKE '%A-01%' OR lp."Lagerplatz" = 'A-01'
|
||||
LIMIT 20
|
||||
|
||||
- Vollständige Artikelinformationen (Preis + Lager):
|
||||
SELECT a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant", e."EP_CHF",
|
||||
lp."Lagerplatz" as "Lagerplatzname", lp."R_LAGER" as "Lager", lp."R_LAGERORT" as "Lagerort",
|
||||
l."S_IST_BESTAND", l."S_SOLL_BESTAND",
|
||||
l."S_MINDESTBESTAND", l."S_MAXIMALBESTAND", l."S_RESERVIERTER__BESTAND",
|
||||
CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
WHERE a."Artikelnummer" = 'ABC123'
|
||||
LIMIT 20
|
||||
|
||||
- Artikel nach Artikelkürzel suchen (z.B. "Wie viele von 141215 haben wir auf Lager?"):
|
||||
SELECT a."Artikelkürzel", a."Artikelnummer", a."Artikelbezeichnung", a."Lieferant",
|
||||
e."EP_CHF", lp."Lagerplatz" as "Lagerplatzname", l."S_IST_BESTAND", l."S_SOLL_BESTAND", l."S_RESERVIERTER__BESTAND",
|
||||
CASE WHEN l."S_IST_BESTAND" != 'Unbekannt' THEN CAST(l."S_IST_BESTAND" AS INTEGER) - COALESCE(l."S_RESERVIERTER__BESTAND", 0) ELSE NULL END as "Verfügbarer Bestand"
|
||||
FROM Artikel a
|
||||
LEFT JOIN Einkaufspreis e ON a."I_ID" = e."m_Artikel"
|
||||
LEFT JOIN Lagerplatz_Artikel l ON a."I_ID" = l."R_ARTIKEL"
|
||||
LEFT JOIN Lagerplatz lp ON l."R_LAGERPLATZ" = lp."I_ID"
|
||||
WHERE a."Artikelkürzel" = '141215'
|
||||
LIMIT 20
|
||||
|
||||
Du hast ausserdem Zugriff auf das Tavily Such-Tool, mit dem du das Internet nach Informationen durchsuchen kannst.
|
||||
Bitte gebrauche das Tool, wenn der Nutzer dich nach mehr informationen zu einem Produkt fragt.
|
||||
Gib auch gerne passende, weiterführende Links an, wenn diese passen.
|
||||
Präferiere offizielle Quellen, möglichst von den Websites der Hersteller selber.
|
||||
Falls du es findest, gib bitte auch einen Link zum offiziellen Produktdatenblatt zurück.
|
||||
|
||||
QUELLENANGABE - INTERNET:
|
||||
WICHTIG: Wenn du Informationen aus dem Internet präsentierst, kennzeichne dies IMMER klar für den Nutzer.
|
||||
- Beginne Internet-Recherchen mit: "Aus meiner Internet-Recherche:" oder "Laut Online-Quellen:"
|
||||
- Gib IMMER die konkreten Quellen an (Website-Namen und Links)
|
||||
- Bei mehreren Quellen: Liste die Quellen auf und verweise darauf
|
||||
- Trenne klar zwischen Datenbank-Informationen und Internet-Recherchen
|
||||
|
||||
Du kannst auch Bilder als Markdown in deiner Antwort einfügen, wenn du dir sicher bist, dass diese die richtigen Bilder zum Produkt sind.
|
||||
Dazu musst du die Bild-URLs anschauen, und auch die Bildbeschreibungen überprüfen.
|
||||
Wenn du dir nicht sicher bist, ob das Bild auch das richtige Produkt zeigt, lasse das Bild weg.
|
||||
Gib in jedem Fall einen kurzen, kleinen Hinweis, dass das Bild möglicherweise vom Produkt abweicht und dann der User sich das Produktdatenblatt ansehen sollte.
|
||||
|
||||
Halluziere keine anderen Fähigkeiten.
|
||||
|
||||
Du antwortest ausschliesslich auf Deutsch. Nutze kein sz(ß) sondern immer ss.
|
||||
|
||||
TABELLEN MIT LAGERBESTÄNDEN - ABSOLUT KRITISCH:
|
||||
JEDE Tabelle, die Lagerbestände zeigt (egal ob nach Artikel, nach Lagerplatz, nach Lieferant oder anders gruppiert), MUSS IMMER folgende Spalten enthalten:
|
||||
- Ist-Bestand (S_IST_BESTAND)
|
||||
- Reservierter Bestand (S_RESERVIERTER__BESTAND) - OBLIGATORISCH!
|
||||
- Verfügbarer Bestand (berechnet) - OBLIGATORISCH!
|
||||
|
||||
VERBOTEN: Tabellen mit Lagerbeständen, die nur Ist-Bestand, Soll-Bestand, Min-Bestand, Max-Bestand zeigen, aber KEINE reservierten Bestände und KEINEN verfügbaren Bestand!
|
||||
VERBOTEN: "Lagerbestände nach Lagerplätzen" Tabellen ohne reservierte Bestände!
|
||||
VERBOTEN: Jede Tabellendarstellung von Lagerbeständen ohne reservierte Bestände und verfügbaren Bestand!
|
||||
|
||||
Beispiel für VERBOTENE Tabellendarstellung:
|
||||
❌ FALSCH:
|
||||
| Lagerplatz | Ist-Bestand | Soll-Bestand | Min-Bestand | Max-Bestand |
|
||||
|------------|-------------|--------------|-------------|-------------|
|
||||
| 6000-089-010 | 0 | 0 | 0 | 0 |
|
||||
| Kanadevia | 3 | 0 | 0 | 0 |
|
||||
|
||||
✓ RICHTIG:
|
||||
| Lagerplatz | Ist-Bestand | Reservierter Bestand | Verfügbarer Bestand | Soll-Bestand | Min-Bestand | Max-Bestand |
|
||||
|------------|-------------|---------------------|---------------------|--------------|-------------|-------------|
|
||||
| 6000-089-010 | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| Kanadevia | 3 | 0 | 3 | 0 | 0 | 0 |
|
||||
|
||||
TABELLENLÄNGE UND ARTIKELANZAHL - KRITISCH:
|
||||
WICHTIG: Zeige MAXIMAL 20 Artikel in Tabellen. Du darfst und sollst aber ausführliche Erklärungen liefern!
|
||||
|
||||
PROAKTIVES DENKEN - BEVOR du Queries ausführst:
|
||||
1. Analysiere die Nutzer-Anfrage: Erwartet der Nutzer eine Übersicht oder Details?
|
||||
2. Bei breiten Anfragen (z.B. "alle Lampen"):
|
||||
- Führe zuerst COUNT() aus, um Gesamtzahl zu ermitteln
|
||||
- Wenn > 20 Treffer: Biete Zusammenfassung + Top 20 an
|
||||
- Oder: Nutze Aggregationen für Übersicht
|
||||
|
||||
STRATEGIE FÜR VIELE TREFFER (> 20):
|
||||
✓ Zeige Zusammenfassung mit Statistiken (Anzahl, Lieferanten, Preisspanne, Kategorien, Lagerbestände)
|
||||
✓ Dann: Tabelle mit den 20 relevantesten/ersten Artikeln
|
||||
✓ Unter der Tabelle: Hinweis dass weitere Artikel existieren
|
||||
✓ Biete Filteroptionen an (nach Lieferant, Preis, Lagerbestand, etc.)
|
||||
|
||||
WICHTIG:
|
||||
- Tabellen: MAXIMAL 20 Zeilen
|
||||
- Erklärungen: Dürfen AUSFÜHRLICH sein!
|
||||
- Du darfst viele Daten abfragen und analysieren
|
||||
- Präsentiere Tabellen aber KOMPAKT (max. 20 Zeilen)
|
||||
- Ergänze mit detaillierten Erklärungen, Statistiken, Zusammenfassungen
|
||||
|
||||
Beispiel einer guten Antwort:
|
||||
"Aus der Datenbank habe ich 147 verschiedene Lampen gefunden [ausführliche Erklärung]. Hier ist eine Übersicht [Statistiken, Kategorien]. Hier sind die ersten 20 Artikel: [Tabelle mit 20 Zeilen]. _Es existieren weitere 127 Artikel. Möchten Sie nach bestimmten Kriterien filtern?_"
|
||||
|
||||
ZAHLEN-PRÜFUNG - ABSOLUT KRITISCH:
|
||||
BEVOR du deine finale Antwort zurückgibst, MUSST du diese Schritte befolgen:
|
||||
|
||||
1. ZÄHLE die TATSÄCHLICHEN Zeilen in deiner finalen Tabelle
|
||||
2. Diese Zahl ist die EINZIGE korrekte Anzahl für deine Antwort
|
||||
3. Verwende diese Zahl KONSISTENT überall in deiner Antwort:
|
||||
- In der Tabellenüberschrift
|
||||
- In Texten unter der Tabelle
|
||||
- In der Zusammenfassung
|
||||
- Überall wo du die Anzahl erwähnst
|
||||
|
||||
VERBOTEN - Inkonsistente Zahlen:
|
||||
❌ FALSCH: "Verfügbare Lampen (50 Artikel)" + "Zeige die ersten 30 Artikel"
|
||||
✓ RICHTIG: "Verfügbare Lampen (30 Artikel)" + "Zeige 30 Artikel"
|
||||
|
||||
❌ FALSCH: Verschiedene Zahlen an verschiedenen Stellen erwähnen
|
||||
✓ RICHTIG: Eine einzige, konsistente Zahl verwenden
|
||||
|
||||
WICHTIG bei mehreren parallelen Queries:
|
||||
- Wenn du mehrere SQL-Abfragen durchführst (z.B. nach "Lampe", "LED", "Beleuchtung")
|
||||
- Kombinierst du die Ergebnisse in EINER Tabelle
|
||||
- Die Anzahl der Zeilen in dieser FINALEN Tabelle ist die korrekte Zahl
|
||||
- NICHT die Summe der einzelnen Query-Ergebnisse!
|
||||
|
||||
Beispiel-Workflow:
|
||||
1. Führe Queries durch → erhalte Ergebnisse
|
||||
2. Kombiniere zu finaler Tabelle → zähle Zeilen (z.B. 30)
|
||||
3. Schreibe Antwort → verwende "30" überall konsistent
|
||||
4. Verifikation → Prüfe nochmals: Steht überall "30"?
|
||||
|
||||
Falls du dem User strukturierte Daten zurückgibst, formatiere sie bitte als Tabelle.
|
||||
WICHTIG! Falls deine Tabelle nur ein Teil der Daten anzeigt, die du gefunden hast, dann vermerke dies bitte in deiner Antwort unter der Tabelle in markdown _italic_.
|
||||
|
||||
Wenn immer du eine Artikelnummer innerhalb einer Tabelle zurückgibst bitte markiere diese als Markdownlink:
|
||||
[ARTIKELNUMMER](/details/ARTIKELNUMMER). ARTIKELNUMMER ist hierbei der Platzhalter, den du ersetzen musst.
|
||||
WICHTIG! Du musst im Link die ARTIKELNUMMER sicher URL-encodieren. Encodiere aber NICHT die Artikelnummer in eckigen Klammern. Also encodiere den Ankertext nicht!
|
||||
Ausserhalb einer Tabelle musst du keine Links auf Artikelnummern setzen.
|
||||
|
||||
Die erste Nachricht das Nutzers ist eine Antwort auf die folgende Nachricht:
|
||||
"Hallo! Ich bin Ihr KI-Assistent für die Materialverwaltung. Wie kann ich Ihnen heute helfen?"
|
||||
"""
|
||||
119
implementation/Chatbot/legacy/router.py
Normal file
119
implementation/Chatbot/legacy/router.py
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
"""Router for chat related endpoints."""
|
||||
|
||||
import logging
|
||||
import time
|
||||
|
||||
from fastapi import HTTPException
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi.responses import StreamingResponse
|
||||
from langchain_anthropic import ChatAnthropic
|
||||
from langchain_core.embeddings import Embeddings as LCEmbeddings
|
||||
from langgraph.checkpoint.sqlite import SqliteSaver
|
||||
|
||||
from src.auth.dependencies import authenticate
|
||||
from src.chat.schemas import (
|
||||
PostChatMessageRequest,
|
||||
PostChatMessageResponse,
|
||||
)
|
||||
from src.common.errors import ErrorResponse
|
||||
from src.dependencies import (
|
||||
get_embeddings,
|
||||
get_chatmodel,
|
||||
get_chatmemory,
|
||||
)
|
||||
|
||||
|
||||
from src.chat import service as chat_service
|
||||
|
||||
# Set up router
|
||||
router = APIRouter()
|
||||
|
||||
# Set up logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/message",
|
||||
response_model=PostChatMessageResponse,
|
||||
responses={
|
||||
200: {"model": PostChatMessageResponse},
|
||||
400: {"model": ErrorResponse},
|
||||
500: {"model": ErrorResponse},
|
||||
},
|
||||
)
|
||||
async def post_message(
|
||||
request: PostChatMessageRequest,
|
||||
embeddings: LCEmbeddings = Depends(get_embeddings),
|
||||
chatmodel: ChatAnthropic = Depends(get_chatmodel),
|
||||
chatmemory: SqliteSaver = Depends(get_chatmemory),
|
||||
username: str = Depends(authenticate),
|
||||
) -> PostChatMessageResponse:
|
||||
"""Endpoint to send a chat message.
|
||||
|
||||
Args:
|
||||
request: The chat message request.
|
||||
embeddings: The embeddings model.
|
||||
chatmodel: The chat model.
|
||||
chatmemory: The chat memory.
|
||||
username: str = Depends(authenticate)
|
||||
|
||||
Returns:
|
||||
The response containing the full chat message history and thread ID.
|
||||
"""
|
||||
logger.info(f"Received message: {request.message} for thread {request.thread}")
|
||||
|
||||
# TODO: Ratelimits / Credits tbd.
|
||||
|
||||
response = await chat_service.post_message(
|
||||
thread_id=request.thread,
|
||||
message=request.message,
|
||||
chatmodel=chatmodel,
|
||||
chatmemory=chatmemory,
|
||||
embeddings=embeddings,
|
||||
)
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@router.post("/message/stream")
|
||||
async def post_message_stream(
|
||||
request: PostChatMessageRequest,
|
||||
embeddings: LCEmbeddings = Depends(get_embeddings),
|
||||
chatmodel: ChatAnthropic = Depends(get_chatmodel),
|
||||
chatmemory: SqliteSaver = Depends(get_chatmemory),
|
||||
username: str = Depends(authenticate),
|
||||
) -> StreamingResponse:
|
||||
"""Endpoint to send a chat message with streaming progress updates.
|
||||
|
||||
Args:
|
||||
request: The chat message request.
|
||||
embeddings: The embeddings model.
|
||||
chatmodel: The chat model.
|
||||
chatmemory: The chat memory.
|
||||
username: str = Depends(authenticate)
|
||||
|
||||
Returns:
|
||||
StreamingResponse with Server-Sent Events for progress updates.
|
||||
"""
|
||||
logger.info(
|
||||
f"Received streaming message: {request.message} for thread {request.thread}"
|
||||
)
|
||||
|
||||
# time.sleep(5) # slight delay to improve UX
|
||||
|
||||
# raise HTTPException(status_code=501, detail="Bitte erneut versuchen.")
|
||||
|
||||
return StreamingResponse(
|
||||
chat_service.post_message_stream(
|
||||
thread_id=request.thread,
|
||||
message=request.message,
|
||||
chatmodel=chatmodel,
|
||||
chatmemory=chatmemory,
|
||||
embeddings=embeddings,
|
||||
),
|
||||
media_type="text/event-stream",
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
},
|
||||
)
|
||||
36
implementation/Chatbot/legacy/schemas.py
Normal file
36
implementation/Chatbot/legacy/schemas.py
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
"""Schemas for chat package."""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class PostChatMessageRequest(BaseModel):
|
||||
"""Request schema for posting a chat message."""
|
||||
|
||||
thread: str = Field(
|
||||
..., example="8237529", description="Unique identifier for the chat thread"
|
||||
)
|
||||
message: str = Field(
|
||||
..., example="Hello, world!", description="The message content"
|
||||
)
|
||||
|
||||
|
||||
class ChatMessageItem(BaseModel):
|
||||
"""Represents a single chat message item for an API response."""
|
||||
|
||||
role: str = Field(
|
||||
..., examples=["human", "assistant"], description="Role of the message sender"
|
||||
)
|
||||
content: str = Field(
|
||||
..., example="Hello, world!", description="The message content"
|
||||
)
|
||||
|
||||
|
||||
class PostChatMessageResponse(BaseModel):
|
||||
"""Response schema when posting a chat message."""
|
||||
|
||||
thread: str = Field(
|
||||
..., example="8237529", description="Unique identifier for the chat thread"
|
||||
)
|
||||
chat_history: list[ChatMessageItem] = Field(
|
||||
..., description="List of messages in the chat thread"
|
||||
)
|
||||
146
implementation/Chatbot/legacy/service.py
Normal file
146
implementation/Chatbot/legacy/service.py
Normal file
|
|
@ -0,0 +1,146 @@
|
|||
"""Service for chat."""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import AsyncIterator, Any, List
|
||||
|
||||
from src.chat.schemas import ChatMessageItem, PostChatMessageResponse
|
||||
from src.chat.domain.chatbot import Chatbot
|
||||
from src.chat import constants as chat_constants
|
||||
|
||||
from langchain_core.messages import HumanMessage, AIMessage
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def post_message(
|
||||
thread_id: str,
|
||||
message: str,
|
||||
chatmodel: any,
|
||||
chatmemory: any,
|
||||
embeddings: any,
|
||||
) -> PostChatMessageResponse:
|
||||
"""Post a chat message to the chatbot and return the response.
|
||||
|
||||
Args:
|
||||
thread_id: The unique identifier for the chat thread.
|
||||
message: The content of the chat message.
|
||||
chatmodel: The chat model to use for generating responses.
|
||||
chatmemory: The chat memory to use for storing conversation history.
|
||||
embeddings: The embeddings model to use for message embeddings.
|
||||
|
||||
Returns:
|
||||
The response containing the full chat message history and thread ID.
|
||||
"""
|
||||
logger.info(f"Received message: {message} for thread {thread_id}")
|
||||
|
||||
# Create chatbot instance
|
||||
chatbot = await Chatbot.create(
|
||||
model=chatmodel,
|
||||
memory=chatmemory,
|
||||
system_prompt=chat_constants.SYSTEM_PROMPT,
|
||||
)
|
||||
|
||||
# Send message to chatbot
|
||||
response = await chatbot.chat(
|
||||
message=message,
|
||||
chat_id=thread_id,
|
||||
)
|
||||
|
||||
# Parse the response to the correct format
|
||||
chat_history = []
|
||||
for message in response:
|
||||
# Determine the role of the message
|
||||
if isinstance(message, HumanMessage):
|
||||
role = "user"
|
||||
elif isinstance(message, AIMessage):
|
||||
role = "assistant"
|
||||
else:
|
||||
continue # Skip any other message types
|
||||
|
||||
# Skip messages that are structured content, such as tool calls.
|
||||
if not isinstance(message.content, str):
|
||||
continue
|
||||
|
||||
# Append message to chat history
|
||||
item = ChatMessageItem(
|
||||
role=role,
|
||||
content=message.content.strip(),
|
||||
)
|
||||
chat_history.append(item)
|
||||
|
||||
return PostChatMessageResponse(thread=thread_id, chat_history=chat_history)
|
||||
|
||||
|
||||
async def post_message_stream(
|
||||
thread_id: str,
|
||||
message: str,
|
||||
chatmodel: Any,
|
||||
chatmemory: Any,
|
||||
embeddings: Any,
|
||||
) -> AsyncIterator[str]:
|
||||
"""Post a chat message to the chatbot and stream progress updates (SSE)."""
|
||||
logger.info(f"Received streaming message: {message} for thread {thread_id}")
|
||||
|
||||
try:
|
||||
chatbot = await Chatbot.create(
|
||||
model=chatmodel,
|
||||
memory=chatmemory,
|
||||
system_prompt=chat_constants.SYSTEM_PROMPT,
|
||||
)
|
||||
|
||||
current_step = None
|
||||
|
||||
async for event in chatbot.stream_events(message=message, chat_id=thread_id):
|
||||
etype = event.get("type")
|
||||
|
||||
# In case we have transient status updates, forward them as-is
|
||||
if etype == "status":
|
||||
current_step = event.get("label")
|
||||
yield f"data: {json.dumps({'type': 'status', 'label': current_step})}\n\n"
|
||||
continue
|
||||
|
||||
# In case we have final response
|
||||
if etype == "final":
|
||||
response_from_event = event.get("response") or {}
|
||||
|
||||
# Use the chat history from the final event (already normalized by stream_events)
|
||||
chat_history_payload = response_from_event.get("chat_history", [])
|
||||
if isinstance(chat_history_payload, list):
|
||||
# Convert to ChatMessageItem (content is already flattened by stream_events)
|
||||
items: List[ChatMessageItem] = []
|
||||
for it in chat_history_payload:
|
||||
role = it.get("role")
|
||||
content = it.get("content", "")
|
||||
if role in ("user", "assistant") and content:
|
||||
items.append(ChatMessageItem(role=role, content=content))
|
||||
|
||||
response = PostChatMessageResponse(
|
||||
thread=thread_id, chat_history=items
|
||||
)
|
||||
# Yield the final response and exit
|
||||
yield f"data: {json.dumps({'type': 'final', 'response': response.model_dump()})}\n\n"
|
||||
return
|
||||
else:
|
||||
# Unexpected payload format - log warning and return empty history
|
||||
logger.warning(
|
||||
f"Unexpected chat_history format in final event: {type(chat_history_payload)}"
|
||||
)
|
||||
response = PostChatMessageResponse(
|
||||
thread=thread_id, chat_history=[]
|
||||
)
|
||||
yield f"data: {json.dumps({'type': 'final', 'response': response.model_dump()})}\n\n"
|
||||
return
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error in streaming chat: {str(e)}", exc_info=True)
|
||||
yield (
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"type": "error",
|
||||
"message": "Ein Fehler ist aufgetreten. Bitte versuchen Sie es erneut.",
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
)
|
||||
134
implementation/Chatbot/legacy/sqlitetool.py
Normal file
134
implementation/Chatbot/legacy/sqlitetool.py
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
"""Tool that allows the chatbot to interact with a remote SQLite database via API (read-only)."""
|
||||
|
||||
import logging
|
||||
import httpx
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, List, Dict, Any
|
||||
from langchain_core.tools import tool
|
||||
from typing import Callable
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class SQLiteTool:
|
||||
"""Remote SQLite database tool for searching articles via API."""
|
||||
|
||||
api_key: str
|
||||
base_url: str
|
||||
|
||||
@classmethod
|
||||
async def create(cls, *, api_key: str, base_url: str) -> "SQLiteTool":
|
||||
"""Factory method to create SQLiteTool instance.
|
||||
|
||||
Args:
|
||||
api_key: API key for authentication
|
||||
base_url: Base URL of the preprocessing query API
|
||||
|
||||
Returns:
|
||||
SQLiteTool instance
|
||||
"""
|
||||
return cls(api_key=api_key, base_url=base_url)
|
||||
|
||||
def get_tool(self) -> Callable[[str], str]:
|
||||
"""Get the configured LangChain tool."""
|
||||
|
||||
@tool("execute_sql")
|
||||
async def execute_sql_query(sql_query: str) -> str:
|
||||
"""Execute a read-only SELECT query on the remote article database.
|
||||
|
||||
Only SELECT statements are allowed. No PRAGMA, INSERT, UPDATE, DELETE, or DDL operations permitted.
|
||||
The database contains one table named "Data" with article information.
|
||||
Your query must reference this table explicitly (e.g., SELECT * FROM Data WHERE ...).
|
||||
Results are limited to 50 rows.
|
||||
|
||||
Args:
|
||||
sql_query: SQLite SELECT query to execute (read-only operations only)
|
||||
|
||||
Returns:
|
||||
The result of the query execution or an error message.
|
||||
"""
|
||||
logger.info(f"Executing SQL query via API: {sql_query}")
|
||||
try:
|
||||
# Check if query is read-only (starts with SELECT)
|
||||
query_upper = sql_query.strip().upper()
|
||||
if not query_upper.startswith("SELECT"):
|
||||
return "Error: Only SELECT queries are allowed. No INSERT, UPDATE, DELETE, or DDL operations permitted."
|
||||
|
||||
# Additional safety checks for potentially harmful operations
|
||||
forbidden_keywords = [
|
||||
"DROP",
|
||||
"CREATE",
|
||||
"ALTER",
|
||||
"INSERT",
|
||||
"UPDATE",
|
||||
"DELETE",
|
||||
"PRAGMA",
|
||||
"ATTACH",
|
||||
"DETACH",
|
||||
]
|
||||
if any(keyword in query_upper for keyword in forbidden_keywords):
|
||||
return "Error: Query contains forbidden keywords. Only SELECT queries are allowed."
|
||||
|
||||
# Make API request
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.post(
|
||||
self.base_url,
|
||||
json={"query": sql_query},
|
||||
headers={"X-DB-API-Key": self.api_key},
|
||||
)
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
|
||||
# Parse API response
|
||||
if not result.get("success"):
|
||||
error_msg = result.get("message", "Unknown error")
|
||||
return f"Query failed: {error_msg}"
|
||||
|
||||
data = result.get("data", [])
|
||||
row_count = result.get("row_count", 0)
|
||||
columns = result.get("columns", [])
|
||||
|
||||
if row_count == 0:
|
||||
return "Query executed successfully but returned no results."
|
||||
|
||||
# Format results
|
||||
results = []
|
||||
for row in data[:50]: # Limit to 50 rows for readability
|
||||
results.append(str(row))
|
||||
|
||||
return (
|
||||
f"Query executed successfully. Returned {row_count} rows (showing first {min(row_count, 50)}):\n"
|
||||
+ "\n".join(results)
|
||||
)
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
return f"API error: HTTP {e.response.status_code} - {e.response.text}"
|
||||
except httpx.RequestError as e:
|
||||
return f"Network error: {str(e)}"
|
||||
except Exception as e:
|
||||
return f"Error executing query: {str(e)}"
|
||||
|
||||
return execute_sql_query
|
||||
|
||||
async def execute_query(self, query: str) -> Dict[str, Any]:
|
||||
"""Execute a raw SQL query via the remote API.
|
||||
|
||||
Args:
|
||||
query: SQL query string
|
||||
|
||||
Returns:
|
||||
Dictionary with query results from the API
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.post(
|
||||
self.base_url,
|
||||
json={"query": query},
|
||||
headers={"X-DB-API-Key": self.api_key},
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
raise Exception(f"Error executing query: {str(e)}")
|
||||
241
implementation/Chatbot/legacy/streaming_helper.py
Normal file
241
implementation/Chatbot/legacy/streaming_helper.py
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
"""Streaming helper utilities for chat message processing and normalization."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, Literal, Mapping, Optional
|
||||
|
||||
from langchain_core.messages import (
|
||||
AIMessage,
|
||||
BaseMessage,
|
||||
HumanMessage,
|
||||
SystemMessage,
|
||||
ToolMessage,
|
||||
)
|
||||
|
||||
Role = Literal["user", "assistant", "system", "tool"]
|
||||
|
||||
|
||||
class ChatStreamingHelper:
|
||||
"""Pure helper methods for streaming and message normalization.
|
||||
|
||||
This class provides static utility methods for converting between different
|
||||
message formats, extracting content, and normalizing message structures
|
||||
for streaming chat applications.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def role_from_message(*, msg: BaseMessage) -> Role:
|
||||
"""Extract the role from a BaseMessage instance.
|
||||
|
||||
Args:
|
||||
msg: The BaseMessage instance to extract the role from.
|
||||
|
||||
Returns:
|
||||
The role as a string literal: "user", "assistant", "system", or "tool".
|
||||
Defaults to "assistant" if the message type is not recognized.
|
||||
|
||||
Examples:
|
||||
>>> from langchain_core.messages import HumanMessage
|
||||
>>> msg = HumanMessage(content="Hello")
|
||||
>>> ChatStreamingHelper.role_from_message(msg=msg)
|
||||
'user'
|
||||
"""
|
||||
if isinstance(msg, HumanMessage):
|
||||
return "user"
|
||||
if isinstance(msg, AIMessage):
|
||||
return "assistant"
|
||||
if isinstance(msg, SystemMessage):
|
||||
return "system"
|
||||
if isinstance(msg, ToolMessage):
|
||||
return "tool"
|
||||
return getattr(msg, "role", "assistant")
|
||||
|
||||
@staticmethod
|
||||
def flatten_content(*, content: Any) -> str:
|
||||
"""Convert complex content structures to plain text.
|
||||
|
||||
This method handles various content formats including strings, lists of
|
||||
content parts, and dictionaries with text fields. It's designed to
|
||||
normalize content from different message sources into a consistent
|
||||
plain text format.
|
||||
|
||||
Args:
|
||||
content: The content to flatten. Can be:
|
||||
- str: Returned as-is after stripping whitespace
|
||||
- list: Each item processed and joined with newlines
|
||||
- dict: Text extracted from "text" or "content" fields
|
||||
- None: Returns empty string
|
||||
- Any other type: Converted to string
|
||||
|
||||
Returns:
|
||||
The flattened content as a plain text string with whitespace stripped.
|
||||
|
||||
Examples:
|
||||
>>> content = [{"type": "text", "text": "Hello"}, {"type": "text", "text": "world"}]
|
||||
>>> ChatStreamingHelper.flatten_content(content=content)
|
||||
'Hello
|
||||
nworld'
|
||||
|
||||
>>> content = {"text": "Simple message"}
|
||||
>>> ChatStreamingHelper.flatten_content(content=content)
|
||||
'Simple message'
|
||||
"""
|
||||
if content is None:
|
||||
return ""
|
||||
if isinstance(content, str):
|
||||
return content.strip()
|
||||
if isinstance(content, list):
|
||||
parts: List[str] = []
|
||||
for part in content:
|
||||
if isinstance(part, dict):
|
||||
if "text" in part and isinstance(part["text"], str):
|
||||
parts.append(part["text"])
|
||||
elif part.get("type") == "text" and isinstance(
|
||||
part.get("text"), str
|
||||
):
|
||||
parts.append(part["text"])
|
||||
elif "content" in part and isinstance(part["content"], str):
|
||||
parts.append(part["content"])
|
||||
else:
|
||||
# Fallback for unknown dictionary structures
|
||||
val = part.get("value")
|
||||
if isinstance(val, str):
|
||||
parts.append(val)
|
||||
else:
|
||||
parts.append(str(part))
|
||||
return "\n".join(p.strip() for p in parts if p is not None)
|
||||
if isinstance(content, dict):
|
||||
if "text" in content and isinstance(content["text"], str):
|
||||
return content["text"].strip()
|
||||
if "content" in content and isinstance(content["content"], str):
|
||||
return content["content"].strip()
|
||||
return str(content).strip()
|
||||
|
||||
@staticmethod
|
||||
def message_to_dict(*, msg: BaseMessage) -> Dict[str, Any]:
|
||||
"""Convert a BaseMessage instance to a dictionary for streaming output.
|
||||
|
||||
This method normalizes BaseMessage instances into a consistent dictionary
|
||||
format suitable for JSON serialization and streaming to clients.
|
||||
|
||||
Args:
|
||||
msg: The BaseMessage instance to convert.
|
||||
|
||||
Returns:
|
||||
A dictionary containing:
|
||||
- "role": The message role (user, assistant, system, tool)
|
||||
- "content": The flattened message content as plain text
|
||||
- "tool_calls": Tool calls if present (optional)
|
||||
- "name": Message name if present (optional)
|
||||
|
||||
Examples:
|
||||
>>> from langchain_core.messages import HumanMessage
|
||||
>>> msg = HumanMessage(content="Hello there")
|
||||
>>> result = ChatStreamingHelper.message_to_dict(msg=msg)
|
||||
>>> result["role"]
|
||||
'user'
|
||||
>>> result["content"]
|
||||
'Hello there'
|
||||
"""
|
||||
payload: Dict[str, Any] = {
|
||||
"role": ChatStreamingHelper.role_from_message(msg=msg),
|
||||
"content": ChatStreamingHelper.flatten_content(
|
||||
content=getattr(msg, "content", "")
|
||||
),
|
||||
}
|
||||
tool_calls = getattr(msg, "tool_calls", None)
|
||||
if tool_calls:
|
||||
payload["tool_calls"] = tool_calls
|
||||
name = getattr(msg, "name", None)
|
||||
if name:
|
||||
payload["name"] = name
|
||||
return payload
|
||||
|
||||
@staticmethod
|
||||
def dict_message_to_dict(*, obj: Mapping[str, Any]) -> Dict[str, Any]:
|
||||
"""Convert a dictionary-shaped message to a normalized dictionary.
|
||||
|
||||
This method handles messages that come from serialized state and are
|
||||
represented as dictionaries rather than BaseMessage instances. It
|
||||
normalizes various dictionary formats into a consistent structure.
|
||||
|
||||
Args:
|
||||
obj: The dictionary-shaped message to convert. Expected to contain
|
||||
fields like "role", "type", "content", "text", etc.
|
||||
|
||||
Returns:
|
||||
A normalized dictionary containing:
|
||||
- "role": The message role (user, assistant, system, tool)
|
||||
- "content": The flattened message content as plain text
|
||||
- "tool_calls": Tool calls if present (optional)
|
||||
- "name": Message name if present (optional)
|
||||
|
||||
Examples:
|
||||
>>> obj = {"type": "human", "content": "Hello"}
|
||||
>>> result = ChatStreamingHelper.dict_message_to_dict(obj=obj)
|
||||
>>> result["role"]
|
||||
'user'
|
||||
>>> result["content"]
|
||||
'Hello'
|
||||
"""
|
||||
role: Optional[str] = obj.get("role")
|
||||
if not role:
|
||||
# Handle alternative type field mappings
|
||||
typ = obj.get("type")
|
||||
if typ in ("human", "user"):
|
||||
role = "user"
|
||||
elif typ in ("ai", "assistant"):
|
||||
role = "assistant"
|
||||
elif typ in ("system",):
|
||||
role = "system"
|
||||
elif typ in ("tool", "function"):
|
||||
role = "tool"
|
||||
|
||||
content = obj.get("content")
|
||||
if content is None and "text" in obj:
|
||||
content = obj["text"]
|
||||
|
||||
out: Dict[str, Any] = {
|
||||
"role": role or "assistant",
|
||||
"content": ChatStreamingHelper.flatten_content(content=content),
|
||||
}
|
||||
if "tool_calls" in obj:
|
||||
out["tool_calls"] = obj["tool_calls"]
|
||||
if obj.get("name"):
|
||||
out["name"] = obj["name"]
|
||||
return out
|
||||
|
||||
@staticmethod
|
||||
def extract_messages_from_output(*, output_obj: Any) -> List[Any]:
|
||||
"""Extract messages from LangGraph output objects.
|
||||
|
||||
This method handles various output formats from LangGraph execution,
|
||||
extracting the messages list from different possible structures.
|
||||
|
||||
Args:
|
||||
output_obj: The output object from LangGraph execution. Can be:
|
||||
- An object with a "messages" attribute
|
||||
- A dictionary with a "messages" key
|
||||
- Any other object (returns empty list)
|
||||
|
||||
Returns:
|
||||
A list of extracted messages, or an empty list if no messages
|
||||
are found or if the output object is None.
|
||||
|
||||
Examples:
|
||||
>>> output = {"messages": [{"role": "user", "content": "Hello"}]}
|
||||
>>> messages = ChatStreamingHelper.extract_messages_from_output(output_obj=output)
|
||||
>>> len(messages)
|
||||
1
|
||||
"""
|
||||
if output_obj is None:
|
||||
return []
|
||||
|
||||
# Try to parse dicts first
|
||||
if isinstance(output_obj, dict):
|
||||
msgs = output_obj.get("messages")
|
||||
return msgs if isinstance(msgs, list) else []
|
||||
|
||||
# Then try to get messages attribute
|
||||
msgs = getattr(output_obj, "messages", None)
|
||||
return msgs if isinstance(msgs, list) else []
|
||||
213
implementation/PLATTFORM_FEATURES_UND_ROADMAP.md
Normal file
213
implementation/PLATTFORM_FEATURES_UND_ROADMAP.md
Normal file
|
|
@ -0,0 +1,213 @@
|
|||
# PowerOn Plattform – Features, Setup & Roadmap
|
||||
|
||||
**Stand:** Februar 2026
|
||||
**Zielgruppe:** Intern, Präsentation, Roadmap-Planung
|
||||
|
||||
---
|
||||
|
||||
## 1. Überblick
|
||||
|
||||
Dieses Dokument beschreibt die vorhandenen Features der PowerOn-Plattform, das allgemeine Plattform-Setup (Security, Mandanten, Nutzerverwaltung) sowie die Einordnung in die Roadmap März–Mai 2026.
|
||||
|
||||
---
|
||||
|
||||
## 2. Allgemeines Plattform-Setup
|
||||
|
||||
### 2.1 Mandanten-/User-Management
|
||||
|
||||
PowerOn ist eine **Multi-Mandanten-Plattform**:
|
||||
|
||||
- **Mandatenmodell:** Jede Organisation/Kunde wird als Mandant abgebildet.
|
||||
- **Datenisolation:** Daten eines Mandanten sind strikt von anderen Mandanten getrennt.
|
||||
- **Zugehörigkeitsprüfung:** Bei jedem Zugriff wird geprüft, ob der Nutzer dem Mandanten angehört (via `UserMandate`).
|
||||
- **Request-Kontext:** `mandateId` und `featureInstanceId` werden über Header (`X-Mandate-Id`, `X-Feature-Instance-Id`) übergeben.
|
||||
- **Mehrfachmandanten:** Nutzer können in mehreren Mandanten tätig sein; der Kontext wird pro Anfrage festgelegt.
|
||||
|
||||
**Zentrale Entitäten:**
|
||||
|
||||
| Entität | Beschreibung |
|
||||
|---------|--------------|
|
||||
| Mandate | Mandant (Organisation, Abteilung, Kunde) |
|
||||
| UserInDB | Nutzer (Benutzername, Profil) |
|
||||
| UserMandate | Zuordnung Nutzer ↔ Mandant (mit enabled) |
|
||||
| UserMandateRole | Rollenzuweisung auf Mandanten-Ebene |
|
||||
| FeatureInstance | Instanz eines Features innerhalb eines Mandanten |
|
||||
| FeatureAccess | Zugriff Nutzer auf Feature-Instanz |
|
||||
| FeatureAccessRole | Rollen pro Feature-Instanz |
|
||||
|
||||
### 2.2 Rollenbasierte Zugriffskontrolle (RBAC)
|
||||
|
||||
- **AccessRule-Kontext:** System, API, DATA
|
||||
- **AccessLevel:** NONE, MY, GROUP, ALL
|
||||
- **Template-Rollen:** Pro Feature können globale Rollenvorlagen definiert werden; bei Erstellung einer Feature-Instanz werden diese kopiert.
|
||||
- **Namespaces:**
|
||||
- `data.uam.*` → Mandanten-UAM
|
||||
- `data.chat.*`, `data.files.*`, `data.automation.*` → nutzer-eigen (MY)
|
||||
- `data.feature.{code}.*` → Mandanten-/Feature-spezifisch
|
||||
|
||||
### 2.3 Authentifizierung
|
||||
|
||||
- **Lokal:** Benutzername + Passwort, JWT
|
||||
- **Microsoft (Azure AD / Entra ID):** SSO
|
||||
- **Google:** SSO
|
||||
- **Token:** HTTP-only Cookies, automatische Erneuerung, Widerruf möglich
|
||||
|
||||
### 2.4 Security-Baseline (Compliance)
|
||||
|
||||
- **Verschlüsselung:** AES (Fernet), PBKDF2-HMAC-SHA256 für Konfiguration
|
||||
- **DSGVO:** Auskunft, Löschung, Datenübertragbarkeit, Berichtigung als Self-Service
|
||||
- **Audit-Trail:** Zugriffe, Sicherheitsereignisse, DSGVO-Aktionen, Berechtigungsänderungen
|
||||
- **OWASP:** CSRF, Rate Limiting, parametrisierte Queries, CORS
|
||||
- **Neutralisierung:** Optionales Modul zur Maskierung/Pseudonymisierung von PII vor KI-Versand
|
||||
|
||||
---
|
||||
|
||||
## 3. Feature-Übersicht (`gateway/modules/features`)
|
||||
|
||||
### 3.1 Chatbot
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `chatbot` |
|
||||
| **Zweck** | KI-gestützter Chat mit Workflows, Tools, Memory |
|
||||
| **DB** | `poweron_chatbot` |
|
||||
| **Besonderheiten** | RBAC pro `featureInstanceId`, streaming, Multi-Round, Tools (z. B. Websuche), Event-Manager |
|
||||
|
||||
### 3.2 Chatplayground
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `chatplayground` |
|
||||
| **Zweck** | Sichere Testumgebung für KI-Chat |
|
||||
| **Besonderheiten** | Wrapper um `interfaceDbChat` mit Feature-Instanz-Kontext; isolierte Umgebung pro Mandant |
|
||||
|
||||
### 3.3 TeamsBot
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `teamsbot` |
|
||||
| **Zweck** | Microsoft Teams Bot mit authentifiziertem Nutzer, Protokollierung |
|
||||
| **DB** | `poweron_teamsbot` |
|
||||
| **Besonderheiten** | Sessions, Transcripts, Bot-Antworten; System-Bots (mandantenbezogen); Browser-Connector für Interaktionen |
|
||||
|
||||
### 3.4 Trustee (BuHa-Integration)
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `trustee` |
|
||||
| **Zweck** | Treuhand/Buchhaltung: Organisationen, Verträge, Dokumente, Positionen, Buchungssynchronisation |
|
||||
| **DB** | `poweron_trustee` |
|
||||
| **Accounting-Connectors** | Bexio, Abacus, RMA |
|
||||
| **Besonderheiten** | Feature-eigene Rollen (admin, operate, userreport), RBAC + feature-spezifische Zugriffslogik |
|
||||
|
||||
### 3.5 Neutralization (Dokumentbearbeitung mit KI)
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `neutralization` |
|
||||
| **Zweck** | Maskierung/Pseudonymisierung sensibler Daten in Dokumenten (PII) vor KI-Verarbeitung |
|
||||
| **DB** | `poweron_neutralization` |
|
||||
| **Besonderheiten** | PDF in-place (PyMuPDF), Platzhalter (z. B. `[name.uuid]`), konfigurierbare Muster; Neutralize Playground |
|
||||
|
||||
### 3.6 Automation
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `automation` |
|
||||
| **Zweck** | Zeitgesteuerte / Event-getriggerte Workflows |
|
||||
| **DB** | `poweron_automation` |
|
||||
| **Besonderheiten** | AutomationDefinition, AutomationTemplate (System + Instanz), Event-Management, Callback-Registry |
|
||||
|
||||
### 3.7 RealEstate
|
||||
|
||||
| Aspekt | Inhalt |
|
||||
|--------|--------|
|
||||
| **Code** | `realestate` |
|
||||
| **Zweck** | Immobilienverwaltung: Projekte, Parzellen, Dokumente, Kanton/Gemeinde, Geo-Daten |
|
||||
| **DB** | `poweron_realestate` |
|
||||
| **Besonderheiten** | BZO-Extraktion, Swiss Topo Scraping, LangGraph |
|
||||
|
||||
---
|
||||
|
||||
## 4. Einordnung der strategischen Features (Roadmap-relevant)
|
||||
|
||||
| Feature | Status | Beschreibung |
|
||||
|---------|--------|--------------|
|
||||
| **Chatbot / Playground** | Live | Sichere Umgebung für Nutzer; Chatplayground als abgegrenzte Testinstanz |
|
||||
| **Private LLM** | Live | Ollama-basierter Connector (`privatellm`); lokal/on-premise; kein Datenabfluss |
|
||||
| **BuHa-Integration** | Live | Trustee mit Connectors Bexio, Abacus, RMA; Sync zu Buchhaltungssystemen |
|
||||
| **TeamsChatbot** | Live | Bot in Teams mit identifiziertem Nutzer; voll protokolliert |
|
||||
| **Dokumentbearbeitung mit KI** | In Arbeit | Neutralization (Masking) + Referenzprozess ERP/Dokument-Extraktion; strukturierter Output |
|
||||
|
||||
---
|
||||
|
||||
## 5. Roadmap März–Mai 2026: Maßnahmenplan
|
||||
|
||||
*Ziel bis Ende Mai: erste produktive Deployments live, Standard-Onboarding v1, Paid Usage gestartet, Security & Compliance als Default.*
|
||||
|
||||
### März 2026 – Foundations + Pilot→Prod
|
||||
|
||||
| Maßnahme | Verantwortlich |
|
||||
|----------|----------------|
|
||||
| Go-Live Readiness Checklist v1 (Monitoring, Logging, Backup/Restore, Incident-Flow, Release-Prozess) | Platform/Engineering |
|
||||
| Mandanten-/User-Setup „v1“ (Roles, Zugriff, Audit-Logs minimum viable) | Security/Compliance |
|
||||
| Security-Baseline aktivieren: Masking/Neutralisierung als Standard pro Workflow + Definition „Datenklasse A/B/C“ | Security/Compliance |
|
||||
| 1 Referenzprozess auswählen & finalisieren (z. B. ERP/Dokument-Extraktion mit strukturiertem Output) | Platform/Engineering |
|
||||
| Erste produktive Installation vorbereiten (Kunde A) | Customer Success |
|
||||
|
||||
**Output Ende März:** 1 produktiver Kunde „go-live ready“, Onboarding-Runbook v1, Security Default aktiv
|
||||
|
||||
---
|
||||
|
||||
### April 2026 – Erste produktive Deployments + Reproduzierbarkeit
|
||||
|
||||
| Maßnahme | Verantwortlich |
|
||||
|----------|----------------|
|
||||
| Produktivsetzung Kunde A inkl. Abnahme (SLA-light, Supportkanal) | Customer Success |
|
||||
| Onboarding v1 standardisieren: Templates (Config, Policies, Integrationsmodule), Setup in < 6 Wochen | Platform/Engineering |
|
||||
| Workflow Engine hardening: Fehlerbehandlung, Retries, Idempotenz, Kontextgrenzen (wichtigste 20 %) | Platform/Engineering |
|
||||
| Kunde B starten (zweites Setup als Wiederholbarkeitstest) | Go-To-Market |
|
||||
| Kosten-/Usage-Metering einführen (Nutzer, Runs, Tokens, Zeitersparnis-Indikator) | Platform/Engineering |
|
||||
|
||||
**Output Ende April:** 1 Kunde produktiv, 2. Kunde im Setup, Onboarding v1 wiederholbar, Usage-Metriken sichtbar
|
||||
|
||||
---
|
||||
|
||||
### Mai 2026 – Paid Usage + Skalierbarer Betrieb
|
||||
|
||||
| Maßnahme | Verantwortlich |
|
||||
|----------|----------------|
|
||||
| Paid Usage aktivieren (Pricing/Package v1, Billing-Prozess) | Go-To-Market |
|
||||
| Support & Betrieb v1: Monitoring-Dashboards, Alerting, Incident-Routinen, Release-Kadenz (z. B. 2-wöchentlich) | Platform/Engineering |
|
||||
| Security & Compliance ausbauen: Policy-Sets pro Workflow, Auditierbarkeit (wer/wann/was), Fallback lokales LLM | Security/Compliance |
|
||||
| Kunde B go-live + Kunde C als Pipeline-Test | Customer Success |
|
||||
| „First Value Moment“ messbar: pro Kunde 1–2 Kennzahlen | Customer Success |
|
||||
|
||||
**Output Ende Mai:** mindestens 2 produktive Deployments, erste zahlende Nutzung, Betrieb stabil, First Value messbar
|
||||
|
||||
---
|
||||
|
||||
## 6. Zuordnung Features ↔ Roadmap
|
||||
|
||||
| Feature | März | April | Mai | Anmerkung |
|
||||
|---------|------|-------|-----|-----------|
|
||||
| Neutralisierung | Security Default aktiv | — | Policy-Sets pro Workflow | Datenklasse A/B/C definieren |
|
||||
| Trustee/BuHa | Referenzprozess ERP/Dok-Extraktion | Wiederholbarkeit Onboarding | — | strukturierter Output |
|
||||
| Chatbot/Playground | — | — | — | bereits produktiv nutzbar |
|
||||
| TeamsBot | — | — | — | authentifizierter User, protokolliert |
|
||||
| Private LLM | — | — | Fallback-Strategie dokumentieren | für höchste Datenschutzanforderungen |
|
||||
|
||||
---
|
||||
|
||||
## 7. Verantwortlichkeiten (Slide-Footer)
|
||||
|
||||
| Rolle | Verantwortungsbereich |
|
||||
|-------|------------------------|
|
||||
| **Platform/Engineering** | Stabilität, Deployment, Monitoring, Workflow-Hardening |
|
||||
| **Security/Compliance** | Policies, Masking, Audit, Datenklassifikation |
|
||||
| **Go-To-Market** | Kundenpipeline, Packaging/Pricing v1, Paid Activation |
|
||||
| **Customer Success** | Onboarding-Runbook, Abnahme, Adoption/Value Tracking |
|
||||
|
||||
---
|
||||
|
||||
*Dokument basiert auf der Gateway-Codebasis (Stand Februar 2026) und dem Compliance-Dokument `poweron_sicherheit_und_compliance.md`.*
|
||||
632
implementation/SharePoint/concept_sharepoint_feature.md
Normal file
632
implementation/SharePoint/concept_sharepoint_feature.md
Normal file
|
|
@ -0,0 +1,632 @@
|
|||
# Concept: SharePoint Site Creator as Gateway Feature
|
||||
|
||||
> Implementation concept for integrating the SharePoint Site Creator (site creation + landing page customization) as a Plug&Play feature in the PowerOn gateway platform.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The SharePoint feature automates two workflows:
|
||||
|
||||
1. **Site Creation** — Create a Microsoft 365 Group (which provisions a SharePoint Team Site), set up a folder structure, register the site in a central Kundenmandate list, and apply a branded homepage.
|
||||
2. **Landing Page Customization** — Compose a page layout from building blocks (text, headers, images, columns, document library previews) and apply it to an existing site's homepage.
|
||||
|
||||
Both workflows are exposed as a single gateway feature (`sharepoint`) with two UI views and corresponding API endpoints, following the existing Plug&Play feature pattern.
|
||||
|
||||
---
|
||||
|
||||
## 2. Feature Identity
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Feature code | `sharepoint` |
|
||||
| Feature folder | `modules/features/sharepoint/` |
|
||||
| Router prefix | `/api/sharepoint` |
|
||||
| Feature label | `{"en": "SharePoint", "de": "SharePoint", "fr": "SharePoint"}` |
|
||||
| Feature icon | `mdi-microsoft-sharepoint` (or `mdi-web`) |
|
||||
|
||||
---
|
||||
|
||||
## 3. File Structure
|
||||
|
||||
Following the established gateway convention (same structure as chatbotV2, realEstate, etc.):
|
||||
|
||||
```
|
||||
modules/features/sharepoint/
|
||||
├── __init__.py
|
||||
├── mainSharepoint.py # FEATURE_CODE, UI/RESOURCE_OBJECTS, TEMPLATE_ROLES, registration
|
||||
├── routeFeatureSharepoint.py # APIRouter(prefix="/api/sharepoint"), route handlers
|
||||
├── interfaceFeatureSharepoint.py # DB access layer (getInterface factory)
|
||||
├── datamodelFeatureSharepoint.py # Pydantic models: SharepointSiteOrder, LandingPageJob
|
||||
├── serviceSharepoint.py # Business logic orchestration (create site flow, customize page flow)
|
||||
├── config.py # SharePointSettings (Pydantic), token cache
|
||||
└── bridges/
|
||||
├── __init__.py
|
||||
├── graphApi.py # Microsoft Graph API client (auth, groups, sites, folders, lists, users)
|
||||
└── pageCustomization.py # Landing page application (PnP subprocess or SP REST API)
|
||||
```
|
||||
|
||||
No changes to `app.py` or `registry.py` are needed — the registry auto-discovers any folder under `modules/features/` that contains a `routeFeature*.py` file.
|
||||
|
||||
**Required one-time changes outside the feature folder:**
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `modules/routes/routeSystem.py` → `_getFeatureUiObjects()` | Add `elif featureCode == "sharepoint"` branch |
|
||||
| `env_int.env` / `env_prod.env` | Add SharePoint environment variables (see Section 9) |
|
||||
| Frontend: `pageRegistry.tsx` | Add icon mappings for `feature.sharepoint`, `page.feature.sharepoint.*` |
|
||||
| Frontend: new view components | `SharepointCreateSiteView.tsx`, `SharepointLandingPageView.tsx` |
|
||||
| Frontend: new API module | `sharepointApi.ts` |
|
||||
| Frontend: new hook | `useSharepoint.ts` |
|
||||
|
||||
---
|
||||
|
||||
## 4. Main Module (`mainSharepoint.py`)
|
||||
|
||||
Defines the feature's identity, RBAC catalog objects, and template roles.
|
||||
|
||||
### 4.1 UI Objects
|
||||
|
||||
Two views — one for creating sites, one for customizing landing pages:
|
||||
|
||||
```python
|
||||
FEATURE_CODE = "sharepoint"
|
||||
FEATURE_LABEL = {"en": "SharePoint", "de": "SharePoint", "fr": "SharePoint"}
|
||||
FEATURE_ICON = "mdi-microsoft-sharepoint"
|
||||
|
||||
UI_OBJECTS = [
|
||||
{
|
||||
"objectKey": "ui.feature.sharepoint.createsite",
|
||||
"label": {"en": "Create Site", "de": "Site erstellen", "fr": "Créer un site"},
|
||||
"meta": {"area": "createsite"}
|
||||
},
|
||||
{
|
||||
"objectKey": "ui.feature.sharepoint.landingpage",
|
||||
"label": {"en": "Landing Page", "de": "Startseite", "fr": "Page d'accueil"},
|
||||
"meta": {"area": "landingpage"}
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
### 4.2 Resource Objects
|
||||
|
||||
```python
|
||||
RESOURCE_OBJECTS = [
|
||||
{
|
||||
"objectKey": "resource.feature.sharepoint.createSite",
|
||||
"label": {"en": "Create SharePoint Site", "de": "SharePoint Site erstellen"},
|
||||
"meta": {"endpoint": "/api/sharepoint/{instanceId}/create-site", "method": "POST"}
|
||||
},
|
||||
{
|
||||
"objectKey": "resource.feature.sharepoint.customizeLandingPage",
|
||||
"label": {"en": "Customize Landing Page", "de": "Startseite anpassen"},
|
||||
"meta": {"endpoint": "/api/sharepoint/{instanceId}/customize-landing-page", "method": "POST"}
|
||||
},
|
||||
{
|
||||
"objectKey": "resource.feature.sharepoint.getSiteStatus",
|
||||
"label": {"en": "Get Site Status", "de": "Site-Status abrufen"},
|
||||
"meta": {"endpoint": "/api/sharepoint/{instanceId}/site-status/{jobId}", "method": "GET"}
|
||||
},
|
||||
{
|
||||
"objectKey": "resource.feature.sharepoint.listOrders",
|
||||
"label": {"en": "List Site Orders", "de": "Site-Bestellungen auflisten"},
|
||||
"meta": {"endpoint": "/api/sharepoint/{instanceId}/orders", "method": "GET"}
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
### 4.3 Template Roles
|
||||
|
||||
```python
|
||||
TEMPLATE_ROLES = [
|
||||
{
|
||||
"roleLabel": "sharepoint-viewer",
|
||||
"description": {"en": "View site orders (read-only)", "de": "Site-Bestellungen ansehen (nur lesen)"},
|
||||
"accessRules": [
|
||||
{"context": "UI", "item": "ui.feature.sharepoint.createsite", "view": True},
|
||||
{"context": "RESOURCE", "item": "resource.feature.sharepoint.listOrders", "view": True},
|
||||
{"context": "RESOURCE", "item": "resource.feature.sharepoint.getSiteStatus", "view": True},
|
||||
{"context": "DATA", "item": None, "view": True, "read": "m", "create": "n", "update": "n", "delete": "n"},
|
||||
]
|
||||
},
|
||||
{
|
||||
"roleLabel": "sharepoint-user",
|
||||
"description": {"en": "Create sites and customize landing pages", "de": "Sites erstellen und Startseiten anpassen"},
|
||||
"accessRules": [
|
||||
{"context": "UI", "item": None, "view": True},
|
||||
{"context": "RESOURCE", "item": None, "view": True},
|
||||
{"context": "DATA", "item": None, "view": True, "read": "m", "create": "m", "update": "m", "delete": "m"},
|
||||
]
|
||||
},
|
||||
{
|
||||
"roleLabel": "sharepoint-admin",
|
||||
"description": {"en": "Full access", "de": "Vollzugriff"},
|
||||
"accessRules": [
|
||||
{"context": "UI", "item": None, "view": True},
|
||||
{"context": "RESOURCE", "item": None, "view": True},
|
||||
{"context": "DATA", "item": None, "view": True, "read": "a", "create": "a", "update": "a", "delete": "a"},
|
||||
]
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
The `registerFeature()` and `_syncTemplateRolesToDb()` functions follow the exact same pattern as `mainChatbotV2.py`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Data Models (`datamodelFeatureSharepoint.py`)
|
||||
|
||||
### 5.1 SharepointSiteOrder
|
||||
|
||||
Tracks each site creation request (stored in the gateway DB for audit/history):
|
||||
|
||||
```python
|
||||
class SharepointSiteOrder(BaseModel):
|
||||
id: Optional[str] = None
|
||||
featureInstanceId: str
|
||||
mandateId: str
|
||||
createdBy: str
|
||||
createdAt: Optional[str] = None
|
||||
|
||||
# Form input
|
||||
projektTitle: str
|
||||
kurzbeschrieb: str
|
||||
mandatsId: str
|
||||
firmenkuerzel: str
|
||||
klassifizierung: str # intern | vertraulich | geheim
|
||||
accountManager: str # email
|
||||
projektLeiter: str # email
|
||||
projektstart: str # ISO date
|
||||
projektende: str # ISO date
|
||||
budget: str
|
||||
|
||||
# Result (filled after creation)
|
||||
status: str = "pending" # pending | provisioning | completed | failed
|
||||
siteUrl: Optional[str] = None
|
||||
groupId: Optional[str] = None
|
||||
siteId: Optional[str] = None
|
||||
warnings: list[str] = []
|
||||
errorMessage: Optional[str] = None
|
||||
```
|
||||
|
||||
### 5.2 LandingPageJob
|
||||
|
||||
Tracks landing page customization requests:
|
||||
|
||||
```python
|
||||
class LandingPageJob(BaseModel):
|
||||
id: Optional[str] = None
|
||||
featureInstanceId: str
|
||||
mandateId: str
|
||||
createdBy: str
|
||||
createdAt: Optional[str] = None
|
||||
|
||||
siteUrl: str
|
||||
pageTitle: str
|
||||
elements: list[dict] # serialized LandingPageElement list
|
||||
status: str = "pending" # pending | processing | completed | failed
|
||||
errorMessage: Optional[str] = None
|
||||
```
|
||||
|
||||
### 5.3 RBAC Table Registration
|
||||
|
||||
Add to `modules/interfaces/interfaceRbac.py` → `TABLE_NAMESPACE`:
|
||||
|
||||
```python
|
||||
"SharepointSiteOrder": "feature.sharepoint",
|
||||
"LandingPageJob": "feature.sharepoint",
|
||||
```
|
||||
|
||||
This ensures `getRecordsetWithRBAC` filters by `featureInstanceId` and applies MY/GROUP/ALL access levels correctly.
|
||||
|
||||
---
|
||||
|
||||
## 6. API Routes (`routeFeatureSharepoint.py`)
|
||||
|
||||
All routes are scoped under `/api/sharepoint/{instanceId}/...` and validate instance access using the same `_validateInstanceAccess` pattern as other features.
|
||||
|
||||
### 6.1 Endpoints
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|--------|------|---------|
|
||||
| `POST` | `/{instanceId}/create-site` | Submit site creation order (multipart/form-data with header image) |
|
||||
| `GET` | `/{instanceId}/orders` | List site orders for this instance (paginated, RBAC-filtered) |
|
||||
| `GET` | `/{instanceId}/orders/{orderId}` | Get status/details of a specific order |
|
||||
| `POST` | `/{instanceId}/customize-landing-page` | Submit landing page customization (multipart/form-data) |
|
||||
| `GET` | `/{instanceId}/landing-page-jobs/{jobId}` | Get status of a landing page job |
|
||||
|
||||
### 6.2 Create Site Endpoint (detail)
|
||||
|
||||
```
|
||||
POST /api/sharepoint/{instanceId}/create-site
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
Fields:
|
||||
projekt_title: str
|
||||
kurzbeschrieb: str
|
||||
mandats_id: str
|
||||
firmenkuerzel: str
|
||||
klassifizierung: str
|
||||
account_manager: str (email)
|
||||
projekt_leiter: str (email)
|
||||
projektstart: str (ISO date)
|
||||
projektende: str (ISO date)
|
||||
budget: str
|
||||
header_image: File (image)
|
||||
|
||||
Response (202 Accepted):
|
||||
{
|
||||
"orderId": "uuid",
|
||||
"status": "provisioning",
|
||||
"message": "Site creation started. Poll GET /orders/{orderId} for status."
|
||||
}
|
||||
```
|
||||
|
||||
The 202 response is intentional: site creation takes 30–60+ seconds (M365 group provisioning), so the endpoint starts the process asynchronously and returns immediately. The client polls the order status endpoint or (future) uses SSE.
|
||||
|
||||
### 6.3 Customize Landing Page Endpoint (detail)
|
||||
|
||||
```
|
||||
POST /api/sharepoint/{instanceId}/customize-landing-page
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
Fields:
|
||||
site_url: str
|
||||
page_title: str
|
||||
elements: str (JSON array of element objects)
|
||||
header_image: File (optional)
|
||||
image_0, image_1, ...: File (content images, indexed to match elements array)
|
||||
|
||||
Response (202 Accepted):
|
||||
{
|
||||
"jobId": "uuid",
|
||||
"status": "processing",
|
||||
"message": "Landing page customization started."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Service Layer (`serviceSharepoint.py`)
|
||||
|
||||
Orchestrates the multi-step site creation and page customization workflows.
|
||||
|
||||
### 7.1 Site Creation Flow
|
||||
|
||||
```
|
||||
async def createSite(user, mandateId, instanceId, formData, headerImage) -> SharepointSiteOrder:
|
||||
|
||||
1. Create SharepointSiteOrder record in DB (status="provisioning")
|
||||
2. Generate site alias from firmenkuerzel + projektTitle
|
||||
3. graphApi.getToken()
|
||||
4. graphApi.createM365Group(alias, title, description)
|
||||
5. graphApi.pollForSiteReady(groupId, timeout=60s, interval=5s)
|
||||
6. graphApi.createFolderStructure(siteId, projektTitle) → warning on failure
|
||||
7. graphApi.addKundenmandateEntry(siteId, formFields) → warning on failure
|
||||
8. pageCustomization.applyHomepageBanner(siteUrl, title, desc, headerImage) → warning on failure
|
||||
9. Update order record: status="completed", siteUrl=..., warnings=[...]
|
||||
10. Return order
|
||||
```
|
||||
|
||||
Steps 6–8 use the **partial success pattern**: if they fail, warnings are recorded but the order is still marked as completed (the site itself exists and is usable). Only steps 4–5 (group creation and site provisioning) are critical failures.
|
||||
|
||||
### 7.2 Landing Page Customization Flow
|
||||
|
||||
```
|
||||
async def customizeLandingPage(user, mandateId, instanceId, siteUrl, pageTitle, elements, images) -> LandingPageJob:
|
||||
|
||||
1. Create LandingPageJob record in DB (status="processing")
|
||||
2. Upload content images to temp storage
|
||||
3. pageCustomization.applyDynamicPage(siteUrl, pageTitle, elements, images)
|
||||
4. Update job record: status="completed" or status="failed"
|
||||
5. Return job
|
||||
```
|
||||
|
||||
### 7.3 Background Execution
|
||||
|
||||
Since both workflows are long-running (30–60s for site creation, 10–30s for page customization), they should run as `asyncio.create_task()` background tasks. The route handler creates the DB record, starts the task, and returns the order/job ID immediately.
|
||||
|
||||
```python
|
||||
@router.post("/{instanceId}/create-site", status_code=202)
|
||||
async def create_site(request: Request, instanceId: str, ...):
|
||||
mandateId = _validateInstanceAccess(instanceId, context)
|
||||
order = _createOrderRecord(...)
|
||||
asyncio.create_task(_executeSiteCreation(order))
|
||||
return {"orderId": order.id, "status": "provisioning"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Bridges
|
||||
|
||||
### 8.1 Microsoft Graph API Bridge (`bridges/graphApi.py`)
|
||||
|
||||
Encapsulates all Graph API calls with token caching and retry logic.
|
||||
|
||||
```python
|
||||
class GraphApiBridge:
|
||||
def __init__(self, tenantId, clientId, clientSecret, tenantName):
|
||||
self._token: str | None = None
|
||||
self._tokenExpiry: float = 0
|
||||
...
|
||||
|
||||
async def getToken(self) -> str: ...
|
||||
async def createM365Group(self, alias, displayName, description) -> str: ...
|
||||
async def pollForSiteReady(self, groupId, timeout=60, interval=5) -> dict: ...
|
||||
async def getSiteDrives(self, siteId) -> list: ...
|
||||
async def createFolder(self, driveId, parentId, name) -> dict: ...
|
||||
async def createFolderStructure(self, siteId, projektTitle) -> dict: ...
|
||||
async def resolveUserByEmail(self, email) -> dict: ...
|
||||
async def getBestellportalSite(self) -> dict: ...
|
||||
async def findKundenmandateList(self, siteId) -> str: ...
|
||||
async def addKundenmandateEntry(self, fields) -> dict: ...
|
||||
```
|
||||
|
||||
Uses `httpx.AsyncClient` for async HTTP. Implements:
|
||||
- Token caching with 5-minute pre-expiry refresh
|
||||
- Exponential backoff retry for 429/503 responses
|
||||
- Configurable timeout for site provisioning polling
|
||||
|
||||
### 8.2 Page Customization Bridge (`bridges/pageCustomization.py`)
|
||||
|
||||
Two implementation options (configurable):
|
||||
|
||||
**Option A: PnP PowerShell with certificate auth** (recommended for MVP)
|
||||
|
||||
```python
|
||||
class PnpPageBridge:
|
||||
async def applyHomepageBanner(self, siteUrl, title, subtitle, headerImagePath): ...
|
||||
async def applyDynamicPage(self, siteUrl, pageTitle, elementsJson, imageDir): ...
|
||||
```
|
||||
|
||||
Invokes PowerShell scripts via `asyncio.create_subprocess_exec`. Uses certificate-based auth (no interactive login). Requires PnP.PowerShell on the server.
|
||||
|
||||
**Option B: SharePoint REST API** (recommended for production, no PowerShell dependency)
|
||||
|
||||
```python
|
||||
class SpRestPageBridge:
|
||||
async def applyHomepageBanner(self, siteUrl, title, subtitle, headerImagePath): ...
|
||||
async def applyDynamicPage(self, siteUrl, pageTitle, elements, images): ...
|
||||
```
|
||||
|
||||
Uses the SharePoint `/_api/sitepages/pages` REST endpoints directly from Python via `httpx`. More complex to implement but eliminates the PowerShell runtime dependency.
|
||||
|
||||
**Decision:** Start with Option A for faster delivery, plan migration to Option B.
|
||||
|
||||
---
|
||||
|
||||
## 9. Configuration
|
||||
|
||||
### 9.1 Environment Variables
|
||||
|
||||
Add to `env_int.env` and `env_prod.env`:
|
||||
|
||||
```env
|
||||
# SharePoint Feature - Azure AD App Registration
|
||||
Feature_Sharepoint_TENANT_ID = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
|
||||
Feature_Sharepoint_CLIENT_ID = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
|
||||
Feature_Sharepoint_CLIENT_SECRET = INT_ENC:...
|
||||
|
||||
# SharePoint Feature - Tenant Info
|
||||
Feature_Sharepoint_TENANT_NAME = contoso.sharepoint.com
|
||||
Feature_Sharepoint_BASE_URL = https://contoso.sharepoint.com
|
||||
|
||||
# SharePoint Feature - Bestellportal
|
||||
Feature_Sharepoint_BESTELLPORTAL_SITE_PATH = /sites/Bestellportal
|
||||
Feature_Sharepoint_KUNDENMANDATE_LIST_NAME = Kundenmandat Bestellungen
|
||||
|
||||
# SharePoint Feature - Provisioning
|
||||
Feature_Sharepoint_PROVISIONING_TIMEOUT = 60
|
||||
Feature_Sharepoint_PROVISIONING_POLL_INTERVAL = 5
|
||||
|
||||
# SharePoint Feature - PnP Auth (if using PowerShell bridge)
|
||||
Feature_Sharepoint_PNP_CERT_PATH = /path/to/cert.pfx
|
||||
Feature_Sharepoint_PNP_CERT_PASSWORD_SECRET = INT_ENC:...
|
||||
```
|
||||
|
||||
### 9.2 Config Class (`config.py`)
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
class SharepointConfig:
|
||||
def __init__(self):
|
||||
self.tenantId = os.getenv("Feature_Sharepoint_TENANT_ID", "")
|
||||
self.clientId = os.getenv("Feature_Sharepoint_CLIENT_ID", "")
|
||||
self.clientSecret = os.getenv("Feature_Sharepoint_CLIENT_SECRET", "")
|
||||
self.tenantName = os.getenv("Feature_Sharepoint_TENANT_NAME", "")
|
||||
self.baseUrl = os.getenv("Feature_Sharepoint_BASE_URL", "")
|
||||
self.bestellportalSitePath = os.getenv("Feature_Sharepoint_BESTELLPORTAL_SITE_PATH", "/sites/Bestellportal")
|
||||
self.kundenmandateListName = os.getenv("Feature_Sharepoint_KUNDENMANDATE_LIST_NAME", "Kundenmandat Bestellungen")
|
||||
self.provisioningTimeout = int(os.getenv("Feature_Sharepoint_PROVISIONING_TIMEOUT", "60"))
|
||||
self.provisioningPollInterval = int(os.getenv("Feature_Sharepoint_PROVISIONING_POLL_INTERVAL", "5"))
|
||||
|
||||
_config = None
|
||||
|
||||
def getSharepointConfig() -> SharepointConfig:
|
||||
global _config
|
||||
if _config is None:
|
||||
_config = SharepointConfig()
|
||||
return _config
|
||||
```
|
||||
|
||||
Follows the same naming convention as existing env vars (`Feature_<Name>_<KEY>`).
|
||||
|
||||
---
|
||||
|
||||
## 10. Frontend Integration
|
||||
|
||||
### 10.1 New Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/api/sharepointApi.ts` | API client (`createSite`, `customizeLandingPage`, `getOrders`, `getOrderStatus`) |
|
||||
| `src/hooks/useSharepoint.ts` | React hooks for state management and polling |
|
||||
| `src/pages/views/sharepoint/SharepointCreateSiteView.tsx` | Site creation form |
|
||||
| `src/pages/views/sharepoint/SharepointLandingPageView.tsx` | Landing page editor |
|
||||
| `src/pages/views/sharepoint/SharepointViews.module.css` | Styles |
|
||||
| `src/pages/views/sharepoint/index.ts` | View exports |
|
||||
|
||||
### 10.2 Page Registry Updates
|
||||
|
||||
In `src/config/pageRegistry.tsx`, add:
|
||||
|
||||
```typescript
|
||||
// Feature pages - SharePoint
|
||||
'page.feature.sharepoint.createsite': <FaBuilding />,
|
||||
'page.feature.sharepoint.landingpage': <FaFileAlt />,
|
||||
|
||||
// Feature icon
|
||||
'feature.sharepoint': <FaBuilding />,
|
||||
```
|
||||
|
||||
### 10.3 FeatureView Mapping
|
||||
|
||||
The `FeatureView.tsx` component maps `uiComponent` codes to React components. Add the sharepoint views following the same pattern as chatbotV2 or trustee views.
|
||||
|
||||
### 10.4 Create Site Form
|
||||
|
||||
The form collects all fields from the SharePoint documentation (Section 5.4):
|
||||
- `projektTitle`, `kurzbeschrieb`, `mandatsId`, `firmenkuerzel`
|
||||
- `klassifizierung` (dropdown: intern/vertraulich/geheim)
|
||||
- `accountManager`, `projektLeiter` (email inputs)
|
||||
- `projektstart`, `projektende` (date pickers)
|
||||
- `budget` (text)
|
||||
- `headerImage` (file upload with preview)
|
||||
|
||||
On submit, sends `multipart/form-data` to `POST /api/sharepoint/{instanceId}/create-site`.
|
||||
|
||||
Shows a progress indicator after submission, polling `GET /orders/{orderId}` every 3 seconds until status is `completed` or `failed`. Displays the site URL on success and any warnings.
|
||||
|
||||
### 10.5 Landing Page Editor
|
||||
|
||||
- Input: existing site URL, page title
|
||||
- Optional header image upload
|
||||
- Element list with add/remove/reorder (drag-and-drop with `@dnd-kit/core`)
|
||||
- Supported element types: text, header, image, columns, files
|
||||
- Each element has a card with type-specific inputs
|
||||
- Preview of element order before submission
|
||||
|
||||
---
|
||||
|
||||
## 11. Integration Touchpoints
|
||||
|
||||
### 11.1 Navigation (`routeSystem.py`)
|
||||
|
||||
Add to `_getFeatureUiObjects()`:
|
||||
|
||||
```python
|
||||
elif featureCode == "sharepoint":
|
||||
from modules.features.sharepoint.mainSharepoint import UI_OBJECTS
|
||||
return UI_OBJECTS
|
||||
```
|
||||
|
||||
This enables the navigation API to build menu entries for SharePoint instances.
|
||||
|
||||
### 11.2 RBAC
|
||||
|
||||
The feature follows the standard RBAC pattern:
|
||||
- Template roles (`sharepoint-viewer`, `sharepoint-user`, `sharepoint-admin`) are synced to DB on startup
|
||||
- When an admin creates a SharePoint feature instance for a mandate, template roles are copied
|
||||
- Route handlers call `_validateInstanceAccess()` before processing
|
||||
- DB queries use `getRecordsetWithRBAC()` to enforce data-level permissions
|
||||
|
||||
### 11.3 Feature Instance Configuration
|
||||
|
||||
When an admin creates a feature instance, the `config` JSON field on the `FeatureInstance` can store instance-specific overrides (e.g., different folder structure template, different Bestellportal path). The service layer reads these from the instance config and falls back to environment variables.
|
||||
|
||||
---
|
||||
|
||||
## 12. Error Handling
|
||||
|
||||
Following the partial success pattern from the SharePoint documentation:
|
||||
|
||||
```
|
||||
Create M365 Group → CRITICAL (fail the order)
|
||||
Poll for Site Ready → CRITICAL (fail with timeout)
|
||||
Create Folder Structure → NON-CRITICAL (add warning, continue)
|
||||
Add Kundenmandate Entry → NON-CRITICAL (add warning, continue)
|
||||
Customize Homepage → NON-CRITICAL (add warning, continue)
|
||||
```
|
||||
|
||||
The `SharepointSiteOrder.warnings` array collects non-critical failure messages. The API response includes these so the frontend can display them.
|
||||
|
||||
For transient Graph API errors (429 rate limiting, 503), the bridge implements exponential backoff with up to 3 retries.
|
||||
|
||||
---
|
||||
|
||||
## 13. Implementation Phases
|
||||
|
||||
### Phase 1 — MVP (Core Site Creation)
|
||||
|
||||
1. Feature skeleton: `mainSharepoint.py`, `routeFeatureSharepoint.py`, `config.py`, `__init__.py`
|
||||
2. Data models and interface
|
||||
3. Graph API bridge: token management, group creation, site polling, folder creation
|
||||
4. `POST /create-site` endpoint (without Kundenmandate list and page customization)
|
||||
5. `GET /orders` and `GET /orders/{orderId}` endpoints
|
||||
6. Frontend: create site form + status polling
|
||||
7. Navigation and RBAC integration
|
||||
|
||||
**Deliverable:** Users can create SharePoint sites with folder structures from the platform.
|
||||
|
||||
### Phase 2 — Kundenmandate & Homepage
|
||||
|
||||
1. Graph API bridge: user lookup, list operations, Kundenmandate entry creation
|
||||
2. PnP bridge: homepage banner application (certificate auth)
|
||||
3. Extend create-site flow with steps 7–8
|
||||
4. Frontend: display warnings from partial success
|
||||
|
||||
**Deliverable:** Full site creation flow including audit list entry and branded homepage.
|
||||
|
||||
### Phase 3 — Landing Page Editor
|
||||
|
||||
1. PnP bridge: dynamic page application
|
||||
2. `POST /customize-landing-page` and `GET /landing-page-jobs/{jobId}` endpoints
|
||||
3. Data model: `LandingPageJob`
|
||||
4. Frontend: landing page editor with drag-and-drop, image upload, element types
|
||||
|
||||
**Deliverable:** Full feature as described in the SharePoint documentation.
|
||||
|
||||
### Phase 4 — Hardening
|
||||
|
||||
1. Replace PnP PowerShell with SharePoint REST API (Option B)
|
||||
2. SSE streaming for real-time progress updates during site creation
|
||||
3. Idempotency check (verify group alias doesn't already exist before creating)
|
||||
4. Configurable folder structure templates per feature instance
|
||||
5. Landing page templates (pre-built layouts users can choose from)
|
||||
|
||||
---
|
||||
|
||||
## 14. Dependencies
|
||||
|
||||
### Backend
|
||||
|
||||
| Package | Purpose | Notes |
|
||||
|---------|---------|-------|
|
||||
| `httpx` | Async HTTP client for Graph API | Already in the project |
|
||||
| `python-multipart` | Form data parsing | Already in the project (FastAPI file uploads) |
|
||||
| PnP.PowerShell (system) | Page customization | Only if using Option A; installed on server OS |
|
||||
|
||||
No new Python packages required for the MVP. The Graph API communication uses `httpx` which is already a project dependency.
|
||||
|
||||
### Frontend
|
||||
|
||||
| Package | Purpose | Notes |
|
||||
|---------|---------|-------|
|
||||
| `@dnd-kit/core` | Drag-and-drop for landing page editor | Phase 3 only; evaluate if already available |
|
||||
|
||||
### Azure AD
|
||||
|
||||
Requires a dedicated App Registration with the permissions listed in the SharePoint documentation Section 3.3 (`Group.ReadWrite.All`, `Sites.FullControl.All`, `Sites.Manage.All`, `User.Read.All`). This can reuse the existing Microsoft service connection (`Service_MSFT_*`) if the required permissions are added, or use a separate registration with `Feature_Sharepoint_*` credentials.
|
||||
|
||||
---
|
||||
|
||||
## 15. Open Questions
|
||||
|
||||
| # | Question | Impact |
|
||||
|---|----------|--------|
|
||||
| 1 | Reuse existing `Service_MSFT_*` credentials or create a dedicated app registration for SharePoint? | Config approach, permission scope |
|
||||
| 2 | Is the Bestellportal site/list already in production, or does it need to be created? | Determines whether Kundenmandate integration is testable from day one |
|
||||
| 3 | Should the folder structure be hardcoded (Arbeitsdokumente/Ergebnisse/Grundlagendokumente) or configurable per instance? | Affects Phase 1 scope |
|
||||
| 4 | Is PnP PowerShell available on the Azure App Service, or should we skip page customization initially? | Determines whether Phase 2 homepage branding is feasible |
|
||||
| 5 | Should site creation progress be streamed via SSE (like chatbot), or is polling sufficient for MVP? | Frontend complexity |
|
||||
BIN
strategy/20260224_Roadmap/20260224_PowerOn_Preismodell.pdf
Normal file
BIN
strategy/20260224_Roadmap/20260224_PowerOn_Preismodell.pdf
Normal file
Binary file not shown.
BIN
strategy/20260224_Roadmap/20260224_PowerOn_Roadmap.pdf
Normal file
BIN
strategy/20260224_Roadmap/20260224_PowerOn_Roadmap.pdf
Normal file
Binary file not shown.
BIN
strategy/20260224_Roadmap/poweron-logo2.png
Normal file
BIN
strategy/20260224_Roadmap/poweron-logo2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
1401
strategy/20260224_Roadmap/roadmap_presentation.html
Normal file
1401
strategy/20260224_Roadmap/roadmap_presentation.html
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Reference in a new issue