84fa0830d8
Expert knows the full eras2_production schema cold: - All PKs, FKs, column names verified from DESCRIBE - Junction tables: objektkunde (kunden↔objekte), objektadressen, kundenadressen - Exact JOIN patterns baked into prompt - No DESCRIBE/SHOW at runtime — plan once, execute - Domain language responses (not SQL dumps) Simplified ExpertNode.execute(): - Removed iterative DESCRIBE→re-plan loop - Single plan+execute pass (schema is known) - Faster: 1 LLM call for plan instead of 2-3 Domain mastery test (eras_domain.md): 38/38 - Customer overview, junction table JOINs, full hierarchy traversal - Address lookup, Verbrauchsdaten, domain language, no DESCRIBE check Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
65 lines
2.3 KiB
Markdown
65 lines
2.3 KiB
Markdown
# Eras Domain Mastery
|
|
|
|
Tests that the expert knows the schema cold — no DESCRIBE at runtime, no SQL errors,
|
|
domain-correct responses. The expert is a Heizkostenabrechnung specialist, not a SQL explorer.
|
|
|
|
## Setup
|
|
- clear history
|
|
|
|
## Steps
|
|
|
|
### 1. Customer overview
|
|
- send: zeig mir die ersten 5 Kunden
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 20
|
|
|
|
### 2. Objekte per Kunde (junction table)
|
|
- send: welcher Kunde hat die meisten Objekte?
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 20
|
|
|
|
### 3. Nutzeinheiten in an Objekt
|
|
- send: wie viele Nutzeinheiten hat Objekt 4?
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 5
|
|
|
|
### 4. Geraete count per Objekt
|
|
- send: welches Objekt hat die meisten Geraete?
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 20
|
|
|
|
### 5. Full hierarchy traversal (4 tables)
|
|
- send: zeig mir alle Nutzer von Kunde 2
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 10
|
|
|
|
### 6. Address lookup via junction
|
|
- send: was ist die Adresse von Objekt 4?
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 10
|
|
|
|
### 7. Verbrauchsdaten query
|
|
- send: zeig mir die letzten 5 Verbrauchswerte von Geraet 100
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: length > 10
|
|
|
|
### 8. Domain language response (not SQL dump)
|
|
- send: gib mir eine Zusammenfassung von Kunde 103
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "SELECT" or "JOIN" or "FROM"
|
|
- expect_response: length > 30
|
|
|
|
### 9. Expert does NOT describe at runtime
|
|
- send: wie viele Geraete hat Kunde 63?
|
|
- expect_trace: has tool_call
|
|
- expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
|
|
- expect_response: not contains "DESCRIBE" or "describe"
|
|
- expect_response: length > 5
|