Add live steering injection into running chains
Let the user inject a message into a running chain instead of only being
able to abort it. While a chain streams, typing + Enter sends the text to
a new /api/chat/steer endpoint; it is drained at the next iteration
boundary, wrapped with a hint so the model actively classifies it
("complements the task -> keep working" vs. "changes course -> adjust"),
and stored as a Type="steer" user message so it survives reload.
Shared infrastructure (provider-agnostic):
- SteeringQueue: out-of-band queue keyed by sessionId, mirrored from
StreamingTracker (Enqueue/Drain/Clear).
- Program.cs registers it as a singleton and passes it to AnthropicService.
- ChatEndpoint: POST /api/chat/steer enqueues only while IsStreaming,
Clear() in the finally.
- ChatService.BuildHistoryFromDb groups Type="steer" so no two
consecutive user turns are produced on re-chat.
- Frontend api.js/chat.js/bubbles.js/history.js + chat.css render the
optimistic steer bubble and the steer_injected SSE event; Enter during
a stream sends a steer instead of being blocked.
Anthropic API path:
- Drain point B (tool boundary, merged into the tool_result user turn)
and C (turn end -> inject + continue instead of finalize).
Claude CLI path (kill-and-restart):
- The agentic loop runs inside the claude child, so true mid-turn inject
is impossible. On a pending steer the child is killed at the next tool
boundary, progress is rebuilt from the DB as a <work_in_progress>
transcript, and the CLI is restarted with the steer appended.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
a58f279eaa
commit
f2fd493e5f
12 files changed
+555
-132
No files matched your search
@@ -19,9 +19,10 @@ public static class ChatEndpoint
|
||||
{
|
||||
app.MapPost("/api/chat", HandleChat);
|
||||
app.MapPost("/api/chat/stop", HandleStop);
|
||||
app.MapPost("/api/chat/steer", HandleSteer);
|
||||
}
|
||||
|
||||
private static async Task HandleChat(HttpRequest req, HttpResponse res, ServerSettingsService settings, ChatService chatService, DatabaseService db, StreamingTracker streamingTracker, AnthropicService anthropic, OpenAiService openAi, RemoteConnectionManager remoteMgr, EmbeddingBackgroundService embeddingBg, CancellationToken cancellationToken)
|
||||
private static async Task HandleChat(HttpRequest req, HttpResponse res, ServerSettingsService settings, ChatService chatService, DatabaseService db, StreamingTracker streamingTracker, SteeringQueue steeringQueue, AnthropicService anthropic, OpenAiService openAi, RemoteConnectionManager remoteMgr, EmbeddingBackgroundService embeddingBg, CancellationToken cancellationToken)
|
||||
{
|
||||
var body = await JsonSerializer.DeserializeAsync<ChatRequest>(req.Body);
|
||||
if (body == null)
|
||||
@@ -277,6 +278,9 @@ public static class ChatEndpoint
|
||||
finally
|
||||
{
|
||||
streamingTracker.Stop(sessionId);
|
||||
// Nach Stop ist IsStreaming(sessionId) false → /api/chat/steer enqueued nichts mehr.
|
||||
// Etwaige nicht gedrainte Einwürfe verwerfen (kein Leak in die nächste Chain).
|
||||
steeringQueue.Clear(sessionId);
|
||||
// Heartbeat-Task beenden (sseLinkCts wird disposed → Token cancelled)
|
||||
}
|
||||
}
|
||||
@@ -297,4 +301,36 @@ public static class ChatEndpoint
|
||||
var stopLevel = streamingTracker.Cancel(sessionId);
|
||||
await res.WriteAsJsonAsync(new { stopped = stopLevel != null, level = stopLevel ?? "none" });
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// "Steering": Reiht eine Nachricht out-of-band in die laufende Chain ein.
|
||||
/// Sie wird an der nächsten Iterations-Grenze als User-Message injiziert.
|
||||
/// Anthropic-API-Pfad: Injektion zwischen den messages.create-Calls.
|
||||
/// Claude-CLI-Pfad: Kill-and-Restart am nächsten Tool-Boundary (der Agentic-Loop
|
||||
/// läuft im claude-Child, daher kein echtes Mid-Turn-Inject möglich).
|
||||
/// </summary>
|
||||
private static async Task HandleSteer(HttpRequest req, HttpResponse res, StreamingTracker streamingTracker, SteeringQueue steeringQueue)
|
||||
{
|
||||
var body = await JsonSerializer.DeserializeAsync<JsonElement>(req.Body);
|
||||
var sessionId = body.TryGetProperty("sessionId", out var sid) ? sid.GetString() : null;
|
||||
var text = body.TryGetProperty("text", out var txt) ? txt.GetString() : null;
|
||||
|
||||
if (string.IsNullOrEmpty(sessionId) || string.IsNullOrWhiteSpace(text))
|
||||
{
|
||||
res.StatusCode = 400;
|
||||
await res.WriteAsJsonAsync(new { error = "sessionId and text required" });
|
||||
return;
|
||||
}
|
||||
|
||||
// Nur einreihen wenn tatsächlich gerade eine Chain läuft – sonst würde der
|
||||
// Einwurf nie gedraint (kein laufender Loop) und bis zur nächsten Chain liegen bleiben.
|
||||
if (!streamingTracker.IsStreaming(sessionId))
|
||||
{
|
||||
await res.WriteAsJsonAsync(new { injected = false, reason = "not_streaming" });
|
||||
return;
|
||||
}
|
||||
|
||||
steeringQueue.Enqueue(sessionId, text.Trim());
|
||||
await res.WriteAsJsonAsync(new { injected = true });
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user