How persistent external knowledge improves frontier agents—and compounds through use
Beakr · Scientific white paper · 16 pages
Research question
Can organizational information be compiled once into a general-purpose external representation—analogous to a foundation model’s internal knowledge, but mutable, source-linked, and shared?
BENCHMARK / HARVEY LAB
More tasks answered completely.
On 250 legal tasks grounded in a law firm’s internal documents, Beakr fully answered 43.8% of tasks versus 35.6% for Codex using the same Beakr knowledge base and rubric judge.
Beakr and Codex share a KB and judge. Engram and Harvey are published context with different judges; scores are not directly comparable.
Rubric criteria met (%)
100
75
50
25
0
Hover, focus, or tap a column to explore.
+8.2 pts
Higher full-pass rate vs Codex
58%
Lower cost per task vs Codex
250
Tasks · Beakr / Codex comparison
Beakr met 85.1% of criteria versus 83.9% for Codex (+1.2 points), while completing more tasks in full at lower cost. Codex cost is API-equivalent. Engram and Harvey use different judges and are shown for context only; the gains above refer exclusively to Beakr versus Codex. The shaded cap on Harvey’s criteria bar marks the published 56.2–63.5% range. Read the results and methodology ↗