agent-runtime/testcases/expert_recovery.md
Nico 067cbccea6 v0.15.8: Expert retry loop, fixed geraete schema, action routing, stable nodes
Expert retry loop (max 3 attempts):
- On SQL error, re-plans with error context injected
- "PREVIOUS ATTEMPTS FAILED" section tells LLM what went wrong
- Breaks out of tool sequence on error, retries full plan
- Only reports failure after exhausting retries
- Recovery test: 13/13

Schema fixes:
- geraete: Geraetenummer, Bezeichnung, Beschreibung (were Fabriknummer, Funkkennung)
- geraeteverbraeuche: all columns verified
- nutzer: all columns verified

Action routing:
- Button clicks route through PA→Expert in v4 (was missing has_pa check)
- WS handler catches exceptions, sends error HUD instead of crashing

Nodes panel:
- Fixed pipeline order, no re-sorting
- Normalized names (pa_v1→pa, expert_eras→eras)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 19:34:01 +02:00

956 B

Expert Recovery

Tests that the expert recovers from SQL errors by retrying with corrected queries, not by reporting the error and stopping.

Setup

  • clear history

Steps

1. Expert recovers from column error silently

  • send: zeig mir alle Geraete von Objekt 4 mit Bezeichnung und Einbaudatum
  • expect_trace: has tool_call
  • expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
  • expect_response: length > 20

2. Multi-table query with potential errors

  • send: zeig mir alle Nutzer und ihre Geraete fuer Kunde 2
  • expect_trace: has tool_call
  • expect_response: not contains "Unknown column" or "1054" or "error" or "Error"
  • expect_response: length > 20

3. Expert does not give up on first failure

  • send: zeig mir Verbrauchswerte fuer Geraet 50 im letzten Monat
  • expect_trace: has tool_call
  • expect_response: not contains "I need assistance" or "developer" or "schema issue"
  • expect_response: length > 10