1. Ingest
~2,000 lines of real system logs per dataset — ~45K–295K tokens each.
We validated Roka against all 16 production log datasets from LogHub (logpai/loghub) — the standard open benchmark for log parsing research. Each sample is ~2,000 lines of real system telemetry.
~2,000 lines of real system logs per dataset — ~45K–295K tokens each.
Fingerprints UUIDs, IPs, thread names, timestamps — repetitive patterns become one line.
BM25 + semantic re-ranking against “what broke” — failures surface first.
Greedy packing to exactly 8,000 tokens — what an agent actually receives.
Most log tools are demoed on tiny, hand-picked snippets. We didn't. We ran the same real-world command an engineer would use during an incident — roka prune --query "what broke" — against every dataset in the
LogHub benchmark suite. These are production logs from Android phones, supercomputers, cloud clusters, and SSH servers — not synthetic toy data.
LogHub is the standard benchmark used in log parsing research (logpai). Each dataset comes from a different system with its own timestamp format, log structure, and failure modes — the same diversity you hit in production.
We didn't tune per dataset. One natural-language query — "what broke" — on all 16 logs. Roka had to auto-detect format, collapse noise, and rank relevance without manual regex rules or per-system config.
Token reduction alone isn't the goal. Every dataset produced a coherent failure narrative: quorum election loss on Zookeeper, brute-force + breach on OpenSSH, CBS errors on Windows. Zero empty outputs.
Why this proves Roka works: If pruning only worked on one log format, you'd see failures across the other 15. Instead, all 16 datasets — mobile, web, HPC, cloud, distributed systems — compress to under 8K tokens while preserving the errors an SRE or AI agent needs to root-cause an incident.
The hard cases (BGL, HPC, Mac, OpenSSH) fill more of the budget because those logs have genuinely diverse failure modes — not because Roka missed anything. When we found format-specific gaps (Spark returning zero output, Thunderbird hostname bloat, Zookeeper election noise), we fixed the pipeline and re-ran until the diagnostic story was correct.
That's the difference between a demo and infrastructure: Roka handles logs you've never seen before, packs them to your token budget, and hands your agent exactly what broke — not 100,000 lines of noise.
Full methodology and Thunderbird write-up: Research →