restructure: flatten project, add ARCHITECTURE.md, update DEVY.md
- Moved from session-analytics-web/ subfolder to root - Renamed to token-analytics - Added deploy.sh, ARCHITECTURE.md - Moved export.mjs into jsonl-sync/
This commit is contained in:
Meleki committed
1 parent
1d3070ed4c
commit
dbaad6b372
17 files changed
+340
-270
No files matched your search
+7
-18
@@ -1,21 +1,10 @@
|
||||
# Node
|
||||
node_modules/
|
||||
|
||||
# .NET
|
||||
bin/
|
||||
obj/
|
||||
/.vs/
|
||||
/bin/
|
||||
/obj/
|
||||
/publish/
|
||||
*.user
|
||||
|
||||
# Data
|
||||
sessions.db
|
||||
state.json
|
||||
/jsonl-sync/node_modules/
|
||||
/jsonl-sync/sessions.db
|
||||
/jsonl-sync/state.json
|
||||
|
||||
# IDE
|
||||
.vs/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
@@ -0,0 +1,51 @@
|
||||
# Architecture
|
||||
|
||||
## Stack
|
||||
|
||||
- **Backend:** ASP.NET (.NET 10), Minimal API
|
||||
- **Frontend:** Vanilla HTML/CSS/JS (SPA, kein Framework)
|
||||
- **Datenbank:** SQLite via Dapper (read) + better-sqlite3 (import)
|
||||
- **Port:** 18684
|
||||
|
||||
## Komponenten
|
||||
|
||||
### Backend (`Program.cs`)
|
||||
Minimal API mit Endpunkten für Kosten-Analyse:
|
||||
- `GET /api/cost-buckets` - Zeitraum in N Buckets aggregiert
|
||||
- `GET /api/cost-entries` - Einzelne Einträge mit Kosten
|
||||
- `GET /api/cost-chains` - Konversations-Ketten nach Kosten sortiert
|
||||
- `GET /api/cost-summary` - Gesamtkosten für Zeitraum
|
||||
- `GET /api/entry-chain/{id}` - Volle Kette eines Eintrags (parentId vor/zurück)
|
||||
- `GET /api/entry-chains` - Einträge mit chainId-Zuordnung
|
||||
- `POST /api/sync` - Triggert JSONL-Import via `export.mjs`
|
||||
|
||||
DB-Zugriff über Dapper + `Microsoft.Data.Sqlite`. Connection per Request via Factory.
|
||||
|
||||
### JSONL Sync (`jsonl-sync/export.mjs`)
|
||||
Node.js Script das OpenClaw Session-JSONL-Files in SQLite importiert:
|
||||
1. Liest alle `.jsonl` aus `/home/node/.openclaw/agents/main/sessions/`
|
||||
2. Parsed Einträge, speichert in `entries` Tabelle
|
||||
3. Berechnet `chainId` durch parentId-Walk bis zum nächsten User-Message
|
||||
4. Inkrementell - merkt sich pro File die letzte verarbeitete Zeile
|
||||
|
||||
### Frontend (`wwwroot/`)
|
||||
SPA mit Zoom-Navigation durch Zeiträume:
|
||||
- **Bucket-View:** Balkendiagramm mit klickbarem Drill-Down (Monat → Woche → Tag → Stunden)
|
||||
- **Entry-View:** Einzelne API-Calls als Balken, farblich nach Konversations-Kette
|
||||
- **Chain-Blocks:** Teuerste Konversationen als klickbare Blöcke
|
||||
- **Modal:** Konversations-Kontext mit User/Assistant Messages
|
||||
- State wird im URL-Hash gespeichert
|
||||
|
||||
## Datenfluss
|
||||
|
||||
```
|
||||
OpenClaw JSONL Files → export.mjs → SQLite (sessions.db) → .NET API → Frontend
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
./deploy.sh
|
||||
```
|
||||
|
||||
Baut Release, kopiert nach `publish/`, restartet den `token-analytics` systemd Service auf stein2.
|
||||
@@ -1,13 +1,21 @@
|
||||
# JSONL Exporter
|
||||
# Token Analytics
|
||||
|
||||
## Projektbeschreibung
|
||||
|
||||
Exportiert OpenClaw Session-JSONL-Dateien als einzelne JSON-Files nach `workspace/logs/`. Inkrementell via `state.json`.
|
||||
Cost-Analytics Dashboard für OpenClaw API-Nutzung. Importiert Session-JSONL-Files in SQLite, bietet eine REST API und ein interaktives Frontend mit Drill-Down von Monats- bis Einzeleintrags-Ebene. Konversations-Ketten werden über parentId-Walks rekonstruiert.
|
||||
|
||||
**Stack:** .NET 10 Minimal API + Vanilla JS SPA + SQLite | **Port:** 18684
|
||||
|
||||
## Pfad
|
||||
|
||||
`/home/node/.openclaw/workspace/projects/jsonl-exporter`
|
||||
`/home/node/.openclaw/workspace/projects/token-analytics`
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
./deploy.sh
|
||||
```
|
||||
|
||||
## Notizen
|
||||
|
||||
-
|
||||
- Siehe ARCHITECTURE.md für Details
|
||||
@@ -1,29 +0,0 @@
|
||||
# JSONL Exporter
|
||||
|
||||
Liest alle Session-JSONL-Dateien aus dem OpenClaw Sessions-Verzeichnis und exportiert jeden Eintrag als einzelne JSON-Datei nach `workspace/logs/`.
|
||||
|
||||
## Funktionsweise
|
||||
|
||||
- Scannt `/home/node/.openclaw/agents/main/sessions/*.jsonl`
|
||||
- Pro Eintrag wird eine Datei mit lesbarem Namen angelegt:
|
||||
`YYYY-MM-DD HH-mm-ss ms - <shortId> (type).json`
|
||||
- Speichert den Fortschritt in `state.json` (nur neue Einträge werden exportiert)
|
||||
- Überspringt gelöschte Sessions (`.deleted`) und fehlerhafte Zeilen
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Zweck |
|
||||
|-------|-------|
|
||||
| `export.mjs` | Hauptskript (ESM) |
|
||||
| `export.js` | Gleiche Logik als CommonJS |
|
||||
| `state.json` | Fortschritts-State (verarbeitete Zeilenzahl pro JSONL) |
|
||||
|
||||
## Ausführen
|
||||
|
||||
```bash
|
||||
node export.mjs
|
||||
# oder
|
||||
node export.js
|
||||
```
|
||||
|
||||
Output landet in: `workspace/logs/`
|
||||
@@ -3,8 +3,7 @@ using Microsoft.Data.Sqlite;
|
||||
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
|
||||
var dbPath = builder.Configuration["DbPath"]
|
||||
?? Path.Combine(AppContext.BaseDirectory, "..", "..", "..", "..", "sessions.db");
|
||||
var dbPath = builder.Configuration["DbPath"]!;
|
||||
|
||||
builder.Services.AddSingleton<Func<SqliteConnection>>(() =>
|
||||
new SqliteConnection($"Data Source={Path.GetFullPath(dbPath)}"));
|
||||
@@ -37,7 +36,6 @@ app.MapGet("/api/cost-buckets", (Func<SqliteConnection> factory, string from, st
|
||||
});
|
||||
|
||||
// GET /api/cost-entries?from=&to=
|
||||
// Returns all cost entries in a time range
|
||||
app.MapGet("/api/cost-entries", (Func<SqliteConnection> factory, string from, string to) =>
|
||||
{
|
||||
using var db = factory();
|
||||
@@ -47,6 +45,7 @@ app.MapGet("/api/cost-entries", (Func<SqliteConnection> factory, string from, st
|
||||
json_extract(data, '$.message.usage.input') AS inputTokens,
|
||||
json_extract(data, '$.message.usage.output') AS outputTokens,
|
||||
json_extract(data, '$.message.model') AS model,
|
||||
chainId,
|
||||
data
|
||||
FROM entries
|
||||
WHERE json_extract(data, '$.message.usage.cost.total') IS NOT NULL
|
||||
@@ -57,6 +56,29 @@ app.MapGet("/api/cost-entries", (Func<SqliteConnection> factory, string from, st
|
||||
return Results.Ok(rows);
|
||||
});
|
||||
|
||||
// GET /api/cost-chains?from=&to=
|
||||
// Returns chains with aggregated cost, sorted by total cost desc
|
||||
app.MapGet("/api/cost-chains", (Func<SqliteConnection> factory, string from, string to, int limit = 50, int offset = 0) =>
|
||||
{
|
||||
using var db = factory();
|
||||
var rows = db.Query(@"
|
||||
SELECT chainId,
|
||||
SUM(CAST(json_extract(data, '$.message.usage.cost.total') AS REAL)) AS totalCost,
|
||||
COUNT(*) AS count,
|
||||
MIN(timestamp) AS first,
|
||||
MAX(timestamp) AS last
|
||||
FROM entries
|
||||
WHERE json_extract(data, '$.message.usage.cost.total') IS NOT NULL
|
||||
AND timestamp >= @from AND timestamp < @to
|
||||
AND chainId IS NOT NULL
|
||||
GROUP BY chainId
|
||||
ORDER BY totalCost DESC
|
||||
LIMIT @limit OFFSET @offset",
|
||||
new { from, to, limit, offset });
|
||||
|
||||
return Results.Ok(rows);
|
||||
});
|
||||
|
||||
// GET /api/cost-summary?from=&to=
|
||||
app.MapGet("/api/cost-summary", (Func<SqliteConnection> factory, string? from, string? to) =>
|
||||
{
|
||||
@@ -151,81 +173,37 @@ app.MapGet("/api/entry-chain/{id}", (string id, Func<SqliteConnection> factory)
|
||||
return Results.Ok(fullChain);
|
||||
});
|
||||
|
||||
// GET /api/entry-chains?from=&to= - returns cost entries with resolved chainId
|
||||
// GET /api/entry-chains?from=&to= - returns cost entries with cached chainId
|
||||
app.MapGet("/api/entry-chains", (Func<SqliteConnection> factory, string from, string to) =>
|
||||
{
|
||||
using var db = factory();
|
||||
|
||||
// Get all cost entries in range
|
||||
var costEntries = db.Query(@"
|
||||
var rows = db.Query(@"
|
||||
SELECT id, session, type, timestamp,
|
||||
json_extract(data, '$.message.usage.cost.total') AS cost,
|
||||
json_extract(data, '$.message.usage.input') AS inputTokens,
|
||||
json_extract(data, '$.message.usage.output') AS outputTokens,
|
||||
json_extract(data, '$.message.model') AS model,
|
||||
json_extract(data, '$.parentId') AS parentId,
|
||||
chainId,
|
||||
data
|
||||
FROM entries
|
||||
WHERE json_extract(data, '$.message.usage.cost.total') IS NOT NULL
|
||||
AND timestamp >= @from AND timestamp < @to
|
||||
ORDER BY timestamp ASC", new { from, to }).AsList();
|
||||
ORDER BY timestamp ASC",
|
||||
new { from, to }).AsList();
|
||||
|
||||
// For each cost entry, walk up parentId chain to find the nearest user message.
|
||||
// That user message ID becomes the chain root.
|
||||
// Entries sharing the same user-message ancestor = same turn/chain.
|
||||
var chainRoot = new Dictionary<string, string>();
|
||||
|
||||
string FindUserRoot(string entryId, SqliteConnection conn)
|
||||
{
|
||||
if (chainRoot.TryGetValue(entryId, out var cached)) return cached;
|
||||
|
||||
var visited = new HashSet<string> { entryId };
|
||||
var currentId = entryId;
|
||||
|
||||
while (true)
|
||||
{
|
||||
var row = conn.QueryFirstOrDefault(
|
||||
@"SELECT json_extract(data, '$.parentId') AS parentId,
|
||||
json_extract(data, '$.message.role') AS role
|
||||
FROM entries WHERE id = @id",
|
||||
new { id = currentId });
|
||||
|
||||
if (row == null) break;
|
||||
|
||||
string role = row.role;
|
||||
// If we landed on a user message, that's our chain root
|
||||
if (role == "user" && currentId != entryId)
|
||||
{
|
||||
chainRoot[entryId] = currentId;
|
||||
return currentId;
|
||||
}
|
||||
|
||||
string parentId = row.parentId;
|
||||
if (parentId == null || visited.Contains(parentId)) break;
|
||||
visited.Add(parentId);
|
||||
currentId = parentId;
|
||||
}
|
||||
|
||||
// No user message found - entry is its own root
|
||||
chainRoot[entryId] = entryId;
|
||||
return entryId;
|
||||
}
|
||||
|
||||
foreach (var e in costEntries) FindUserRoot((string)e.id, db);
|
||||
|
||||
// Assign chain index by first appearance
|
||||
// Map chainId to sequential index (for coloring)
|
||||
var chainIndex = new Dictionary<string, int>();
|
||||
int nextChain = 0;
|
||||
var result = costEntries.Select(e =>
|
||||
var result = rows.Select(e =>
|
||||
{
|
||||
var root = chainRoot[(string)e.id];
|
||||
if (!chainIndex.ContainsKey(root)) chainIndex[root] = nextChain++;
|
||||
string cid = e.chainId ?? e.id;
|
||||
if (!chainIndex.ContainsKey(cid)) chainIndex[cid] = nextChain++;
|
||||
return new
|
||||
{
|
||||
e.id, e.session, e.type, e.timestamp,
|
||||
e.cost, e.inputTokens, e.outputTokens, e.model,
|
||||
e.data,
|
||||
chainId = chainIndex[root]
|
||||
chainId = chainIndex[cid]
|
||||
};
|
||||
});
|
||||
|
||||
@@ -235,12 +213,12 @@ app.MapGet("/api/entry-chains", (Func<SqliteConnection> factory, string from, st
|
||||
// POST /api/sync - run export.mjs to import new entries
|
||||
app.MapPost("/api/sync", async () =>
|
||||
{
|
||||
var scriptPath = Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "..", "..", "..", "..", "export.mjs"));
|
||||
var psi = new System.Diagnostics.ProcessStartInfo("node", scriptPath)
|
||||
var wd = "/home/node/.openclaw/workspace/projects/token-analytics/jsonl-sync";
|
||||
var psi = new System.Diagnostics.ProcessStartInfo("docker", $"exec meleki node {wd}/export.mjs")
|
||||
{
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
WorkingDirectory = Path.GetDirectoryName(scriptPath)!
|
||||
WorkingDirectory = wd!
|
||||
};
|
||||
var proc = System.Diagnostics.Process.Start(psi)!;
|
||||
var stdout = await proc.StandardOutput.ReadToEndAsync();
|
||||
@@ -7,5 +7,5 @@
|
||||
},
|
||||
"AllowedHosts": "*",
|
||||
"Urls": "http://0.0.0.0:18684",
|
||||
"DbPath": "../sessions.db"
|
||||
"DbPath": "jsonl-sync/sessions.db"
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
export PATH="$HOME/.dotnet:$PATH"
|
||||
cd ~/.openclaw/workspace/projects/token-analytics
|
||||
|
||||
dotnet publish -c Release -o publish
|
||||
|
||||
|
||||
ssh -o ConnectTimeout=5 meleki@192.168.100.200 "nohup sudo systemctl restart token-analytics > /dev/null 2>&1 &"
|
||||
echo "Service restarted"
|
||||
-81
@@ -1,81 +0,0 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import Database from 'better-sqlite3';
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const SESSIONS_DIR = '/home/node/.openclaw/agents/main/sessions';
|
||||
const DB_FILE = path.join(__dirname, 'sessions.db');
|
||||
|
||||
// DB öffnen + Schema anlegen
|
||||
const db = new Database(DB_FILE);
|
||||
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS entries (
|
||||
id TEXT PRIMARY KEY,
|
||||
session TEXT NOT NULL,
|
||||
type TEXT,
|
||||
timestamp TEXT,
|
||||
data TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_session ON entries (session);
|
||||
CREATE INDEX IF NOT EXISTS idx_type ON entries (type);
|
||||
CREATE INDEX IF NOT EXISTS idx_timestamp ON entries (timestamp);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS state (
|
||||
file TEXT PRIMARY KEY,
|
||||
lines INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
`);
|
||||
|
||||
const getState = db.prepare('SELECT lines FROM state WHERE file = ?');
|
||||
const setState = db.prepare('INSERT OR REPLACE INTO state (file, lines) VALUES (?, ?)');
|
||||
const insertEntry = db.prepare(`
|
||||
INSERT OR IGNORE INTO entries (id, session, type, timestamp, data)
|
||||
VALUES (@id, @session, @type, @timestamp, @data)
|
||||
`);
|
||||
|
||||
// Find all JSONL files
|
||||
const jsonlFiles = fs.readdirSync(SESSIONS_DIR)
|
||||
.filter(f => f.endsWith('.jsonl') && !f.includes('.deleted'));
|
||||
|
||||
let exported = 0;
|
||||
|
||||
const runImport = db.transaction(() => {
|
||||
for (const file of jsonlFiles) {
|
||||
const filePath = path.join(SESSIONS_DIR, file);
|
||||
const sessionName = file.replace('.jsonl', '');
|
||||
const lines = fs.readFileSync(filePath, 'utf-8').split('\n').filter(l => l.trim());
|
||||
|
||||
const lastProcessed = getState.get(file)?.lines ?? 0;
|
||||
|
||||
for (let i = lastProcessed; i < lines.length; i++) {
|
||||
try {
|
||||
const entry = JSON.parse(lines[i]);
|
||||
if (!entry.timestamp) continue;
|
||||
|
||||
const id = entry.id || `${sessionName}-${i}`;
|
||||
|
||||
insertEntry.run({
|
||||
id,
|
||||
session: sessionName,
|
||||
type: entry.type ?? 'unknown',
|
||||
timestamp: entry.timestamp,
|
||||
data: JSON.stringify(entry),
|
||||
});
|
||||
|
||||
exported++;
|
||||
} catch {
|
||||
// skip malformed lines
|
||||
}
|
||||
}
|
||||
|
||||
setState.run(file, lines.length);
|
||||
}
|
||||
});
|
||||
|
||||
runImport();
|
||||
db.close();
|
||||
|
||||
console.log(`Exportiert: ${exported} neue Einträge → ${DB_FILE}`);
|
||||
@@ -0,0 +1,140 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import Database from 'better-sqlite3';
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const SESSIONS_DIR = '/home/node/.openclaw/agents/main/sessions';
|
||||
const DB_FILE = path.join(__dirname, 'sessions.db');
|
||||
|
||||
const db = new Database(DB_FILE);
|
||||
|
||||
// Schema
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS entries (
|
||||
id TEXT PRIMARY KEY,
|
||||
session TEXT NOT NULL,
|
||||
type TEXT,
|
||||
timestamp TEXT,
|
||||
data TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_session ON entries (session);
|
||||
CREATE INDEX IF NOT EXISTS idx_type ON entries (type);
|
||||
CREATE INDEX IF NOT EXISTS idx_timestamp ON entries (timestamp);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS state (
|
||||
file TEXT PRIMARY KEY,
|
||||
lines INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
`);
|
||||
|
||||
// Add chainId column if missing (migration)
|
||||
try { db.exec('ALTER TABLE entries ADD COLUMN chainId TEXT'); } catch {}
|
||||
try { db.exec('CREATE INDEX IF NOT EXISTS idx_chainId ON entries (chainId)'); } catch {}
|
||||
|
||||
const getState = db.prepare('SELECT lines FROM state WHERE file = ?');
|
||||
const setState = db.prepare('INSERT OR REPLACE INTO state (file, lines) VALUES (?, ?)');
|
||||
const insertEntry = db.prepare(`
|
||||
INSERT OR IGNORE INTO entries (id, session, type, timestamp, data)
|
||||
VALUES (@id, @session, @type, @timestamp, @data)
|
||||
`);
|
||||
const getEntry = db.prepare('SELECT id, data FROM entries WHERE id = ?');
|
||||
const setChainId = db.prepare('UPDATE entries SET chainId = @chainId WHERE id = @id');
|
||||
const getUnchained = db.prepare(`
|
||||
SELECT id, data FROM entries
|
||||
WHERE chainId IS NULL
|
||||
ORDER BY timestamp ASC
|
||||
`);
|
||||
|
||||
// --- Step 1: Import new entries ---
|
||||
const jsonlFiles = fs.readdirSync(SESSIONS_DIR)
|
||||
.filter(f => f.endsWith('.jsonl') && !f.includes('.deleted'));
|
||||
|
||||
let exported = 0;
|
||||
|
||||
const runImport = db.transaction(() => {
|
||||
for (const file of jsonlFiles) {
|
||||
const filePath = path.join(SESSIONS_DIR, file);
|
||||
const sessionName = file.replace('.jsonl', '');
|
||||
const lines = fs.readFileSync(filePath, 'utf-8').split('\n').filter(l => l.trim());
|
||||
const lastProcessed = getState.get(file)?.lines ?? 0;
|
||||
|
||||
for (let i = lastProcessed; i < lines.length; i++) {
|
||||
try {
|
||||
const entry = JSON.parse(lines[i]);
|
||||
if (!entry.timestamp) continue;
|
||||
const id = entry.id || `${sessionName}-${i}`;
|
||||
insertEntry.run({
|
||||
id,
|
||||
session: sessionName,
|
||||
type: entry.type ?? 'unknown',
|
||||
timestamp: entry.timestamp,
|
||||
data: JSON.stringify(entry),
|
||||
});
|
||||
exported++;
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
|
||||
setState.run(file, lines.length);
|
||||
}
|
||||
});
|
||||
|
||||
runImport();
|
||||
|
||||
// --- Step 2: Compute chainId for entries that don't have one ---
|
||||
// chainId = the nearest user message when walking up the parentId chain.
|
||||
// If no user message found, chainId = own id.
|
||||
|
||||
let chained = 0;
|
||||
const chainCache = new Map(); // id -> chainId
|
||||
|
||||
function resolveChainId(id) {
|
||||
if (chainCache.has(id)) return chainCache.get(id);
|
||||
|
||||
const visited = new Set();
|
||||
let currentId = id;
|
||||
|
||||
while (currentId && !visited.has(currentId)) {
|
||||
visited.add(currentId);
|
||||
|
||||
if (chainCache.has(currentId) && currentId !== id) {
|
||||
// Already resolved - reuse
|
||||
const result = chainCache.get(currentId);
|
||||
chainCache.set(id, result);
|
||||
return result;
|
||||
}
|
||||
|
||||
const row = getEntry.get(currentId);
|
||||
if (!row) break;
|
||||
|
||||
const d = JSON.parse(row.data);
|
||||
|
||||
// If this is a user message, it's the chain root
|
||||
if (d.message?.role === 'user') {
|
||||
chainCache.set(id, currentId);
|
||||
return currentId;
|
||||
}
|
||||
|
||||
currentId = d.parentId || null;
|
||||
}
|
||||
|
||||
// No user message found - self is root
|
||||
chainCache.set(id, id);
|
||||
return id;
|
||||
}
|
||||
|
||||
const runChaining = db.transaction(() => {
|
||||
const unchained = getUnchained.all();
|
||||
|
||||
for (const row of unchained) {
|
||||
const chainId = resolveChainId(row.id);
|
||||
setChainId.run({ id: row.id, chainId });
|
||||
chained++;
|
||||
}
|
||||
});
|
||||
|
||||
runChaining();
|
||||
db.close();
|
||||
|
||||
console.log(`Exportiert: ${exported} neue, ${chained} verkettet → ${DB_FILE}`);
|
||||
File renamed without changes.
File renamed without changes.
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/launchsettings.json",
|
||||
"profiles": {
|
||||
"http": {
|
||||
"commandName": "Project",
|
||||
"dotnetRunMessages": true,
|
||||
"launchBrowser": true,
|
||||
"applicationUrl": "http://localhost:5169",
|
||||
"environmentVariables": {
|
||||
"ASPNETCORE_ENVIRONMENT": "Development"
|
||||
}
|
||||
},
|
||||
"https": {
|
||||
"commandName": "Project",
|
||||
"dotnetRunMessages": true,
|
||||
"launchBrowser": true,
|
||||
"applicationUrl": "https://localhost:7160;http://localhost:5169",
|
||||
"environmentVariables": {
|
||||
"ASPNETCORE_ENVIRONMENT": "Development"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,9 +0,0 @@
|
||||
{
|
||||
"DetailedErrors": true,
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"Microsoft.AspNetCore": "Warning"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,9 +4,14 @@
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<RootNamespace>session_analytics_web</RootNamespace>
|
||||
<RootNamespace>token_analytics</RootNamespace>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Content Remove="publish/**" />
|
||||
<None Remove="publish/**" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Dapper" Version="2.1.66" />
|
||||
<PackageReference Include="Microsoft.Data.Sqlite" Version="10.0.3" />
|
||||
@@ -34,9 +34,31 @@ async function init() {
|
||||
viewFrom.setDate(viewFrom.getDate() - ZOOM_LEVELS[0].days);
|
||||
zoomIndex = 0;
|
||||
|
||||
restoreFromHash();
|
||||
await loadView();
|
||||
}
|
||||
|
||||
function saveToHash() {
|
||||
const state = {
|
||||
f: viewFrom.toISOString(),
|
||||
t: viewTo.toISOString(),
|
||||
z: zoomIndex,
|
||||
e: entryMode ? 1 : 0,
|
||||
};
|
||||
location.hash = btoa(JSON.stringify(state));
|
||||
}
|
||||
|
||||
function restoreFromHash() {
|
||||
if (!location.hash || location.hash.length < 2) return;
|
||||
try {
|
||||
const state = JSON.parse(atob(location.hash.slice(1)));
|
||||
viewFrom = new Date(state.f);
|
||||
viewTo = new Date(state.t);
|
||||
zoomIndex = state.z ?? 0;
|
||||
entryMode = !!state.e;
|
||||
} catch {}
|
||||
}
|
||||
|
||||
// --- Main load ---
|
||||
async function loadView() {
|
||||
const from = viewFrom.toISOString();
|
||||
@@ -63,10 +85,14 @@ async function loadView() {
|
||||
api(`/api/cost-summary?from=${enc(from)}&to=${enc(to)}`),
|
||||
]);
|
||||
|
||||
const chains = await api(`/api/cost-chains?from=${enc(from)}&to=${enc(to)}`);
|
||||
|
||||
renderSummary(summary);
|
||||
renderBucketChart(buckets, bucketMinutes);
|
||||
renderBlocks(buckets, bucketMinutes);
|
||||
renderChainBlocks(chains);
|
||||
}
|
||||
|
||||
saveToHash();
|
||||
}
|
||||
|
||||
function renderSummary(s) {
|
||||
@@ -188,46 +214,42 @@ function renderEntryChart(entries) {
|
||||
});
|
||||
}
|
||||
|
||||
// --- Blocks (normal zoom) ---
|
||||
function renderBlocks(buckets, bucketMinutes) {
|
||||
// --- Chain blocks (grouped by conversation chain, sorted by cost) ---
|
||||
function renderChainBlocks(chains) {
|
||||
const container = $('blocks');
|
||||
container.innerHTML = '';
|
||||
|
||||
const bucketMap = {};
|
||||
buckets.forEach(b => { bucketMap[b.bucket] = b; });
|
||||
|
||||
const sortedBuckets = [];
|
||||
for (let i = 0; i < BAR_COUNT; i++) {
|
||||
const b = bucketMap[i];
|
||||
if (!b || b.cost === 0) continue;
|
||||
sortedBuckets.push({ ...b, idx: i });
|
||||
if (!chains.length) {
|
||||
container.innerHTML = '<p style="color:var(--muted);padding:40px;text-align:center">Keine Einträge mit Kosten in diesem Zeitraum.</p>';
|
||||
return;
|
||||
}
|
||||
sortedBuckets.sort((a, b) => b.cost - a.cost);
|
||||
|
||||
for (const { idx: i, ...b } of sortedBuckets) {
|
||||
const bucketStart = new Date(viewFrom.getTime() + i * bucketMinutes * 60000);
|
||||
const bucketEnd = new Date(viewFrom.getTime() + (i + 1) * bucketMinutes * 60000);
|
||||
|
||||
chains.forEach((c, ci) => {
|
||||
const block = document.createElement('div');
|
||||
block.className = 'block';
|
||||
block.innerHTML = `
|
||||
<div class="block-header">
|
||||
<span class="block-label">${fmtRange(bucketStart, bucketEnd)}</span>
|
||||
<span class="block-label">${fmtShort(new Date(c.first))} - ${fmtShort(new Date(c.last))}</span>
|
||||
<span class="block-stats">
|
||||
<span class="block-count">${b.count} Einträge</span>
|
||||
<span class="block-cost">$${b.cost.toFixed(4)}</span>
|
||||
<span class="block-count">${c.count} Einträge</span>
|
||||
<span class="block-cost">$${c.totalCost.toFixed(4)}</span>
|
||||
</span>
|
||||
</div>
|
||||
<div class="block-entries" id="be-${i}"></div>
|
||||
`;
|
||||
block.querySelector('.block-header').addEventListener('click', () =>
|
||||
toggleBlock(i, bucketStart, bucketEnd));
|
||||
block.querySelector('.block-header').addEventListener('click', () => {
|
||||
// Jump directly to entry mode, centered on this chain
|
||||
const first = new Date(c.first);
|
||||
const last = new Date(c.last);
|
||||
const pad = Math.max((last - first) * 0.5, 60000); // min 1 min padding
|
||||
zoomStack.push({ from: new Date(viewFrom), to: new Date(viewTo), zoom: zoomIndex, entry: false });
|
||||
viewFrom = new Date(first.getTime() - pad);
|
||||
viewTo = new Date(last.getTime() + pad);
|
||||
zoomIndex = ZOOM_LEVELS.length - 1;
|
||||
entryMode = true;
|
||||
loadView();
|
||||
});
|
||||
container.appendChild(block);
|
||||
}
|
||||
|
||||
if (!container.children.length) {
|
||||
container.innerHTML = '<p style="color:var(--muted);padding:40px;text-align:center">Keine Einträge mit Kosten in diesem Zeitraum.</p>';
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// --- Entry list (flat, sorted by cost) ---
|
||||
@@ -259,22 +281,29 @@ function renderEntryList(entries) {
|
||||
});
|
||||
}
|
||||
|
||||
// --- Toggle block entries ---
|
||||
async function toggleBlock(idx, from, to) {
|
||||
const el = $(`be-${idx}`);
|
||||
// --- Toggle chain block entries ---
|
||||
async function toggleChainBlock(idx, chainId) {
|
||||
const el = $(`chain-block-${idx}`);
|
||||
if (el.classList.contains('open')) { el.classList.remove('open'); return; }
|
||||
|
||||
if (!el.dataset.loaded) {
|
||||
const entries = await api(`/api/cost-entries?from=${enc(from.toISOString())}&to=${enc(to.toISOString())}`);
|
||||
el.innerHTML = entries.map(e => `
|
||||
<div class="entry" data-json='${escAttr(e.data)}'>
|
||||
<span class="entry-time">${fmtTime(e.timestamp)}</span>
|
||||
<span class="entry-model">${e.model || '-'}</span>
|
||||
<span class="entry-tokens">${shortNum(e.inputTokens)}/${shortNum(e.outputTokens)}</span>
|
||||
<span class="entry-cost">$${Number(e.cost).toFixed(4)}</span>
|
||||
<span class="entry-type">${e.type}</span>
|
||||
</div>
|
||||
`).join('');
|
||||
// Load the full chain via entry-chain endpoint (using chainId = user message id)
|
||||
const chain = await api(`/api/entry-chain/${encodeURIComponent(chainId)}`);
|
||||
el.innerHTML = chain.map(entry => {
|
||||
const d = JSON.parse(entry.data);
|
||||
const msg = d.message;
|
||||
if (!msg) return '';
|
||||
const cost = msg.usage?.cost?.total;
|
||||
return `
|
||||
<div class="entry" data-json='${escAttr(entry.data)}'>
|
||||
<span class="entry-time">${fmtTime(entry.timestamp)}</span>
|
||||
<span class="entry-model">${msg.model || '-'}</span>
|
||||
<span class="entry-tokens">${shortNum(msg.usage?.input)}/${shortNum(msg.usage?.output)}</span>
|
||||
<span class="entry-cost">${cost ? '$' + cost.toFixed(4) : '-'}</span>
|
||||
<span class="entry-type">${msg.role || entry.type}</span>
|
||||
</div>
|
||||
`;
|
||||
}).filter(Boolean).join('');
|
||||
el.querySelectorAll('.entry').forEach(row => {
|
||||
const eid = JSON.parse(row.dataset.json).id;
|
||||
row.addEventListener('click', () => openModal(row.dataset.json, eid));
|
||||
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Session Cost Analytics</title>
|
||||
<title>Token Analytics</title>
|
||||
<link rel="stylesheet" href="style.css" />
|
||||
</head>
|
||||
<body>
|
||||
@@ -125,7 +125,7 @@ header h1 { font-size: 1.1rem; font-weight: 600; color: var(--accent); }
|
||||
.bar {
|
||||
width: 100%;
|
||||
min-width: 4px;
|
||||
background: var(--green);
|
||||
background: #5b7a9d;
|
||||
border-radius: 3px 3px 0 0;
|
||||
opacity: 0.7;
|
||||
transition: opacity 0.15s, background 0.15s;
|
||||
@@ -133,7 +133,7 @@ header h1 { font-size: 1.1rem; font-weight: 600; color: var(--accent); }
|
||||
|
||||
.bar-wrapper:hover .bar {
|
||||
opacity: 1;
|
||||
background: #6ee7b7;
|
||||
background: #7a9dc0;
|
||||
}
|
||||
|
||||
.bar-tooltip {
|
||||
@@ -147,7 +147,7 @@ header h1 { font-size: 1.1rem; font-weight: 600; color: var(--accent); }
|
||||
font-size: 0.7rem;
|
||||
white-space: nowrap;
|
||||
z-index: 10;
|
||||
color: var(--green);
|
||||
color: var(--text);
|
||||
font-weight: 600;
|
||||
pointer-events: none;
|
||||
}
|
||||
@@ -195,6 +195,7 @@ header h1 { font-size: 1.1rem; font-weight: 600; color: var(--accent); }
|
||||
}
|
||||
|
||||
.block-header:hover { background: #22263a; }
|
||||
.block-header .block-label { pointer-events: none; }
|
||||
|
||||
.block-label {
|
||||
font-family: monospace;
|
||||
Reference in new issue
Block a user