Development material
Cases are selected for internal development and may have been exposed to the project. Small samples and repeated calls do not establish general performance.
PALO BlueprintBench / Local development study
Review agent diagnostics
Local models for source-fidelity review.
Explore how model configurations classify claims against supplied source excerpts. Compare retained responses, processing times and the hardware used to run each test. The advisory agent is intended to challenge draft interpretations before human review.
These tests use the agent prompt and output validator via a local model endpoint. The full agent application workflow was not executed. Model agreement is not ground truth; independent expert adjudication and validation remain pending.
Study limitationsSource fidelity / Model and serving configuration
Results explorer
Filter by configuration and case. Missing and invalid responses remain visible. Numeric sorting is for inspection, not a qualified ranking.
Observed request duration
Median time by case
One row represents a case and a model configuration. Filter to one case when inspecting different configurations. Output caps and prompt identifiers are listed with each result.
Loading observations...
Bars use the maximum visible median as the scale. Different prompts, output lengths, serving settings and hardware affect duration.
Measurement notes
What the results show
- Assignment
- Requests allocated to each case, including missing responses.
- Contract checks
- Complete responses that meet the required format and source quotation checks.
- Agreement
- Comparison with a prior model output. It is not an expert-derived answer key.
- Duration
- Observed end-to-end time in the recorded configuration, not isolated hardware performance.
Case-level observations
Configurations and results
Scroll horizontally to see all fields.
| Configuration / hardware | Case | Output cap / prompt | Completed / assigned | Contract valid | Not completed / invalid | Median time | Completion tokens | Observed label | Agreement | Evidence |
|---|---|---|---|---|---|---|---|---|---|---|
| Loading observations... | ||||||||||
Not completed includes failed, incomplete and unrun calls. Invalid counts refer to completed calls that failed contract checks. A missing duration or label is shown as N/A. No energy use, GPU offload or monetary cost is inferred from a local run.
Study design
Interpretation and limitations
Results belong to the reported model, prompt, runtime and hardware configuration.
Advisory assessment
These tests exercise the agent prompt and output validator via a local model endpoint, not the full agent application workflow. Labels and explanations are model-generated suggestions; matching quotations do not verify their conclusions.
Independent review
Human expertise, independent adjudication, external custody and measurement validation remain separate requirements.