1000411eb2
Frame Engine (v3-framed): - Tick-based deterministic pipeline: frames advance on completion, not timers - FrameRecord/FrameTrace dataclasses for structured per-message tracing - /api/frames endpoint: queryable frame trace history (last 20 messages) - frame_trace HUD event with full pipeline visibility - Reflex=2F, Director=4F, Director+Interpreter=5F deterministic frame counts Expert Architecture (v4-eras): - PA node (pa_v1): routes to domain experts, holds user context - ExpertNode base: stateless executor with plan+execute two-LLM-call pattern - ErasExpertNode: eras2_production DB specialist with DESCRIBE-first discipline - Schema caching: DESCRIBE results reused across queries within session - Progress streaming: PA streams thinking message, expert streams per-tool progress - PARouting type for structured routing decisions UI Controls Split: - Separate thinker_controls from machine controls (current_controls is now a property) - Machine buttons persist across Thinker responses - Machine state parser handles both dict and list formats from Director - Normalized button format with go/payload field mapping WebSocket Architecture: - /ws/test: dedicated debug socket for test runner progress - /ws/trace: dedicated debug socket for HUD/frame trace events - /ws (chat): cleaned up, only deltas/controls/done/cleared - WS survives graph switch (re-attaches to new runtime) - Pipeline result reset on clear Test Infrastructure: - Live test streaming: on_result callback fires per check during execution - Frontend polling fallback (500ms) for proxy-buffered WS - frame_trace-first trace assertion (fixes stale perceived event bug) - action_match supports "or" patterns and multi-pattern matching - Trace window increased to 40 events - Graph-agnostic assertions (has X or Y) Test Suites: - smoketest.md: 12 steps covering all categories (~2min) - fast.md: 10 quick checks (~1min) - fast_v4.md: 10 v4-eras specific checks - expert_eras.md: eras domain tests (routing, DB, schema, errors) - expert_progress.md: progress streaming tests Other: - Shared db.py extracted from thinker_v2 (reused by experts) - InputNode prompt: few-shot examples, history as context summary - Director prompt: full tool signatures for add_state/reset_machine/destroy_machine - nginx no-cache headers for static files during development - Cache-busted static file references Scores: v3 smoketest 39/40, v4-eras fast 28/28, expert_eras 23/23 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
72 lines
2.3 KiB
Markdown
72 lines
2.3 KiB
Markdown
# Smoketest
|
|
|
|
Fast validation: one example per category, covers all 11 suite areas in ~2 minutes.
|
|
|
|
## Setup
|
|
- clear history
|
|
|
|
## Steps
|
|
|
|
### 1. Reflex path (social/trivial skips Thinker)
|
|
- send: hi there!
|
|
- expect_response: length > 3
|
|
- expect_trace: input.analysis.intent is "social"
|
|
- expect_trace: input.analysis.complexity is "trivial"
|
|
|
|
### 2. Input analysis (German detection + question intent)
|
|
- send: Wie spaet ist es?
|
|
- expect_trace: input.analysis.language is "de"
|
|
- expect_trace: input.analysis.intent is "question"
|
|
- expect_response: length > 5
|
|
|
|
### 3. Frustrated tone detection
|
|
- send: this is broken, nothing works and I'm sick of it
|
|
- expect_trace: input.analysis.tone is "frustrated" or "urgent"
|
|
- expect_response: length > 10
|
|
|
|
### 4. Button creation
|
|
- send: create two buttons: Alpha and Beta
|
|
- expect_actions: length >= 2
|
|
- expect_actions: any action contains "alpha" or "Alpha"
|
|
- expect_actions: any action contains "beta" or "Beta"
|
|
|
|
### 5. Dashboard feedback (Thinker sees buttons)
|
|
- send: what buttons can you see in my dashboard?
|
|
- expect_response: contains "Alpha" or "alpha" or "Beta" or "beta"
|
|
|
|
### 6. DB query (tool call + table)
|
|
- send: show me 3 customers from the database
|
|
- expect_trace: has tool_call
|
|
- expect_response: length > 10
|
|
|
|
### 7. Director plan (complex request)
|
|
- send: investigate which customers have the most devices in the database
|
|
- expect_trace: has director_plan or decided
|
|
- expect_trace: has tool_call
|
|
- expect_response: length > 20
|
|
|
|
### 8. Memorizer state (facts + language tracking)
|
|
- send: My dog's name is Bella
|
|
- expect_state: facts any contains "Bella"
|
|
- expect_state: language is "en" or "mixed"
|
|
|
|
### 9. Machine creation
|
|
- send: create a navigation machine called "test" with initial state "ready" showing a Go button
|
|
- expect_trace: has tool_call create_machine
|
|
- expect_trace: machine_created id="test"
|
|
|
|
### 10. Counter with buttons
|
|
- send: create a counter starting at 0 with increment and decrement buttons
|
|
- expect_response: contains "counter" or "count" or "0" or "Zähler"
|
|
- expect_actions: length >= 2
|
|
|
|
### 11. Language switch
|
|
- send: Hallo, wie geht es dir?
|
|
- expect_state: language is "de" or "mixed"
|
|
- expect_response: length > 5
|
|
|
|
### 12. Expert routing (v4 only, safe to skip on v3)
|
|
- send: show me 3 customers from the database
|
|
- expect_trace: has tool_call
|
|
- expect_response: length > 10
|