All updates

External Knowledge for Scientific Agents

How persistent external knowledge improves frontier agents—and compounds through use

Beakr · Scientific white paper · 16 pages

Research question

Can organizational information be compiled once into a general-purpose external representation—analogous to a foundation model’s internal knowledge, but mutable, source-linked, and shared?

BENCHMARK / HARVEY LAB

More tasks answered completely.

On 250 legal tasks grounded in a law firm’s internal documents, Beakr fully answered 43.8% of tasks versus 35.6% for Codex using the same Beakr knowledge base and rubric judge.

Beakr and Codex share a KB and judge. Engram and Harvey are published context with different judges; scores are not directly comparable.

Rubric criteria met (%)

Hover, focus, or tap a column to explore.

+8.2 pts

Higher full-pass rate vs Codex

58%

Lower cost per task vs Codex

250

Tasks · Beakr / Codex comparison

Beakr met 85.1% of criteria versus 83.9% for Codex (+1.2 points), while completing more tasks in full at lower cost. Codex cost is API-equivalent. Engram and Harvey use different judges and are shown for context only; the gains above refer exclusively to Beakr versus Codex. The shaded cap on Harvey’s criteria bar marks the published 56.2–63.5% range. Read the results and methodology ↗