01
Overview
Enterprise knowledge is fragmented across surfaces.
Real workspace questions rarely announce where their answer lives. An agent may need policy text from a document, exact values from a table, and relationships from a dependency graph—sometimes in the same task.
WorkSurface-Bench isolates this routing problem while retaining executable, traceable evidence. It scores the route, retrieved evidence, final answer, and efficiency separately, making failures easier to diagnose than a single accuracy number.
Core question
Can an agent identify and combine the right knowledge surfaces before answering?
02
Knowledge surfaces
Three evidence forms, one shared workspace.
Documents / RAG
Policies and prose
Retrieve grounded spans from unstructured files and supporting documents.
Structured tables
Records and computation
Filter, join, aggregate, and compare exact values from tabular data.
Dependency graphs
Relations and impact
Traverse dependencies, ownership links, and downstream effects.
03
Construction
A filtered, auditable construction pipeline.
- 1Ground task scenarios
Start from workspace scenarios and provenance-locked source files.
- 2Compose atomic tasks
Create single- and cross-surface questions with traceable gold evidence.
- 3Screen automatically
Apply deterministic checks and multi-model quality screening.
- 4Audit with humans
Three annotators independently review a stratified 200-item sample.
04
Evaluation protocol
Decompose success instead of hiding it.
Route F1
Were the necessary knowledge surfaces selected?
Evidence
Was the required supporting evidence actually acquired?
Answer
Is the final response correct under the task rubric?
Efficiency
Did the agent avoid unnecessary tool calls and detours?
Aggregate
0.35A + 0.30E + 0.25R + 0.10Q
05
Main results
Better routing helps, but does not guarantee a correct answer.
Spearman correlation between Route F1 and answer accuracy across model–setting–task buckets.
Route F1 under gold-constrained routing, showing that the routing interface is executable.
Answer accuracy in the same setting: evidence composition and reasoning remain substantial bottlenecks.
Four model backbones evaluated under six controlled agent settings.
06
Dataset composition
Broad coverage without forcing artificial uniformity.
RAG only213
Table only279
Graph only171
RAG + Graph314
Graph + Table100
RAG + Table59
All three15
✓
Human audit: 200/200 items passed by majority vote, with 192 fully unanimous and 98.7–100% pairwise agreement.
07
Get started
Load the benchmark in one line.
Python
from datasets import load_dataset
dataset = load_dataset(
"lhpku20010120/WorkSurface-Bench"
)