Add live steering injection into running chains

Let the user inject a message into a running chain instead of only being
able to abort it. While a chain streams, typing + Enter sends the text to
a new /api/chat/steer endpoint; it is drained at the next iteration
boundary, wrapped with a hint so the model actively classifies it
("complements the task -> keep working" vs. "changes course -> adjust"),
and stored as a Type="steer" user message so it survives reload.

Shared infrastructure (provider-agnostic):
- SteeringQueue: out-of-band queue keyed by sessionId, mirrored from
  StreamingTracker (Enqueue/Drain/Clear).
- Program.cs registers it as a singleton and passes it to AnthropicService.
- ChatEndpoint: POST /api/chat/steer enqueues only while IsStreaming,
  Clear() in the finally.
- ChatService.BuildHistoryFromDb groups Type="steer" so no two
  consecutive user turns are produced on re-chat.
- Frontend api.js/chat.js/bubbles.js/history.js + chat.css render the
  optimistic steer bubble and the steer_injected SSE event; Enter during
  a stream sends a steer instead of being blocked.

Anthropic API path:
- Drain point B (tool boundary, merged into the tool_result user turn)
  and C (turn end -> inject + continue instead of finalize).

Claude CLI path (kill-and-restart):
- The agentic loop runs inside the claude child, so true mid-turn inject
  is impossible. On a pending steer the child is killed at the next tool
  boundary, progress is rebuilt from the DB as a <work_in_progress>
  transcript, and the CLI is restarted with the steer appended.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MelekiandClaude Opus 4.8 committed 2026-06-02 06:49:58 +02:00
1 parent a58f279eaa
commit f2fd493e5f
12 files changed
+555 -132

No files matched your search

+37 -1
View File
@@ -19,9 +19,10 @@ public static class ChatEndpoint
{
app.MapPost("/api/chat", HandleChat);
app.MapPost("/api/chat/stop", HandleStop);
app.MapPost("/api/chat/steer", HandleSteer);
}
private static async Task HandleChat(HttpRequest req, HttpResponse res, ServerSettingsService settings, ChatService chatService, DatabaseService db, StreamingTracker streamingTracker, AnthropicService anthropic, OpenAiService openAi, RemoteConnectionManager remoteMgr, EmbeddingBackgroundService embeddingBg, CancellationToken cancellationToken)
private static async Task HandleChat(HttpRequest req, HttpResponse res, ServerSettingsService settings, ChatService chatService, DatabaseService db, StreamingTracker streamingTracker, SteeringQueue steeringQueue, AnthropicService anthropic, OpenAiService openAi, RemoteConnectionManager remoteMgr, EmbeddingBackgroundService embeddingBg, CancellationToken cancellationToken)
{
var body = await JsonSerializer.DeserializeAsync<ChatRequest>(req.Body);
if (body == null)
@@ -277,6 +278,9 @@ public static class ChatEndpoint
finally
{
streamingTracker.Stop(sessionId);
// Nach Stop ist IsStreaming(sessionId) false → /api/chat/steer enqueued nichts mehr.
// Etwaige nicht gedrainte Einwürfe verwerfen (kein Leak in die nächste Chain).
steeringQueue.Clear(sessionId);
// Heartbeat-Task beenden (sseLinkCts wird disposed → Token cancelled)
}
}
@@ -297,4 +301,36 @@ public static class ChatEndpoint
var stopLevel = streamingTracker.Cancel(sessionId);
await res.WriteAsJsonAsync(new { stopped = stopLevel != null, level = stopLevel ?? "none" });
}
/// <summary>
/// "Steering": Reiht eine Nachricht out-of-band in die laufende Chain ein.
/// Sie wird an der nächsten Iterations-Grenze als User-Message injiziert.
/// Anthropic-API-Pfad: Injektion zwischen den messages.create-Calls.
/// Claude-CLI-Pfad: Kill-and-Restart am nächsten Tool-Boundary (der Agentic-Loop
/// läuft im claude-Child, daher kein echtes Mid-Turn-Inject möglich).
/// </summary>
private static async Task HandleSteer(HttpRequest req, HttpResponse res, StreamingTracker streamingTracker, SteeringQueue steeringQueue)
{
var body = await JsonSerializer.DeserializeAsync<JsonElement>(req.Body);
var sessionId = body.TryGetProperty("sessionId", out var sid) ? sid.GetString() : null;
var text = body.TryGetProperty("text", out var txt) ? txt.GetString() : null;
if (string.IsNullOrEmpty(sessionId) || string.IsNullOrWhiteSpace(text))
{
res.StatusCode = 400;
await res.WriteAsJsonAsync(new { error = "sessionId and text required" });
return;
}
// Nur einreihen wenn tatsächlich gerade eine Chain läuft – sonst würde der
// Einwurf nie gedraint (kein laufender Loop) und bis zur nächsten Chain liegen bleiben.
if (!streamingTracker.IsStreaming(sessionId))
{
await res.WriteAsJsonAsync(new { injected = false, reason = "not_streaming" });
return;
}
steeringQueue.Enqueue(sessionId, text.Trim());
await res.WriteAsJsonAsync(new { injected = true });
}
}