| What it does | Test LLMs on your own tasks before you switch models
| Human evaluation for text-to-speech with blinded listening studies
|
|---|
| Stage | Not given
| Not given
|
|---|
| Founded | Not given
| Not given
|
|---|
| Based in | Not given
| US
|
|---|
| Pricing model | Paid
| Paid
|
|---|
| Pricing | Standard: $99 one-time for 2 machines, including 1 year of updates. Pro coming soon.
| $18.00 per hour of audio heard by listeners; $0.30 per judgement on a pair of two 30-second clips; $300.00 for 200 pairs × 5 ratings with 30-second clips (1,000 judgements)
|
|---|
| Key features | - Benchmark any OpenRouter model on your own prompts, CSVs or Hugging Face datasets
- Rubric-based scoring by AI judge, by hand, or both
- Cost per answered task with retries and caching
- Time to first token and output tokens per second per call
- Shareable reports in HTML, CSV and JSON formats
- Compare up to three past benchmarks side by side
- Hard spending cap per run
- Fully self-hosted on your machine
- No SDK or code changes required
| - Blinded paired-preference studies with no system names visible
- API for creating studies and retrieving results
- Exact price quote before study starts
- Per-judgement data export as CSV
- Paid US-based listener panel
- Attention signals recorded with every judgement
- Live results streaming as listeners answer
- Detection template for AI vs human narration
- Published methodology and platform transparency
|
|---|
| What makes it different | - Tests your own tasks with real data instead of generic benchmarks
- Comes before integration with no SDK or code changes needed
- Reports actual cost per answered task including retries and caching
- Entirely self-hosted with data never leaving your machine
- One shareable report with every answer and judge reasoning
| - Publishes its own listening studies on the same platform with the same methodology as customer studies
- Forced-choice preference testing answers which system people prefer, not rating scales
- Fast turnaround with live result streaming and no minimum study size
- Transparent pricing with no setup fees or minimums, quoted before starting
- Customer audio stored securely and used only for that study
|
|---|
| Community votes | 0 | 0 |
|---|
| DR (Domain Rating by Ahrefs) | Not checked yet ● no change since last check | 8 ● no change since last check |
|---|
| Trust Flow (Majestic) | 0 | 9 |
|---|
| Citation Flow (Majestic) | 16 | 24 |
|---|
| Referring domains (Majestic) | 3 | 69 |
|---|
orbench
Woodstitch