| What it does | Test LLMs on your own tasks before you switch models
| Independent verification for AI agent outcomes
|
|---|
| Stage | Not given
| Startup
|
|---|
| Founded | Not given
| Not given
|
|---|
| Based in | Not given
| Not given
|
|---|
| Pricing model | Paid
| Freemium
|
|---|
| Pricing | Standard: $99 one-time for 2 machines, including 1 year of updates. Pro coming soon.
| Hobbyist: free forever with 250 checks/month; Pro: $9/month with 2,500 checks/month, $0.01 per extra check
|
|---|
| Key features | - Benchmark any OpenRouter model on your own prompts, CSVs or Hugging Face datasets
- Rubric-based scoring by AI judge, by hand, or both
- Cost per answered task with retries and caching
- Time to first token and output tokens per second per call
- Shareable reports in HTML, CSV and JSON formats
- Compare up to three past benchmarks side by side
- Hard spending cap per run
- Fully self-hosted on your machine
- No SDK or code changes required
| - Independent outcome verification with VERIFIED, FAILED, or UNKNOWN status
- First-party adapters for Stripe, GitHub, Linear, Sentry, Resend, and HTTP endpoints
- MCP integration with Claude Code, Cursor, and any MCP host
- GitHub Actions required check that fails on unverified claims
- Audit trail storing claims, evidence, and verdicts separately
- Capability inspection before agents call tools
- Read-only credential access with AES-256-GCM encryption
- Continuous monitoring with watches every 5-10 minutes on Pro plan
- Evidence retention for 7 days on Hobbyist, 1 year on Pro
|
|---|
| What makes it different | - Tests your own tasks with real data instead of generic benchmarks
- Comes before integration with no SDK or code changes needed
- Reports actual cost per answered task including retries and caching
- Entirely self-hosted with data never leaving your machine
- One shareable report with every answer and judge reasoning
| - Refuses to guess: returns UNKNOWN rather than false positives when evidence is unreachable
- Neutral gate between agent execution and success verification
- Runs the same way in production as in development
- Never needs write access, only reads authoritative state
- Structured audit trail for debugging and agent evaluation
|
|---|
| Community votes | 0 | 0 |
|---|
| DR (Domain Rating by Ahrefs) | Not checked yet ● no change since last check | 2 ● no change since last check |
|---|
| Trust Flow (Majestic) | 0 | 1 |
|---|
| Citation Flow (Majestic) | 16 | 23 |
|---|
| Referring domains (Majestic) | 3 | 14 |
|---|
orbench
DidWork