REST API

Tally provides a REST API for programmatic access to your projects, corpora, predicates, rubrics, runs, and evaluations. Everything you can do in the UI is available via the API.

Authentication

All API requests require a Bearer token in the Authorization header.

Creating a token

1. Go to Settings > API Tokens
2. Enter a label (e.g. "CI pipeline") and optional expiry
3. Click Create
4. Copy the token immediately — it won't be shown again

Using a token

Include the token in every request:

curl -H "Authorization: Bearer tly_your_token_here" \
     https://tally.pythia.software/api/projects

Project context

Most endpoints are scoped to a project. Set the project via the X-Project-Id header:

curl -H "Authorization: Bearer tly_..." \
     -H "X-Project-Id: your-project-uuid" \
     https://tally.pythia.software/api/corpus

If omitted, your default project is used.

Quick start

1. List your projects

curl -H "Authorization: Bearer $TOKEN" \
     $TALLY_URL/api/projects

2. Create a corpus

curl -X POST \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -H "X-Project-Id: $PROJECT_ID" \
     -d '{"name": "My Corpus", "description": "Test content"}' \
     $TALLY_URL/api/corpus

3. Create a predicate

curl -X POST \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -H "X-Project-Id: $PROJECT_ID" \
     -d '{
       "name": "Is Clear",
       "preamble": "You are evaluating text for clarity.",
       "question": "Is this text written clearly and easy to understand?"
     }' \
     $TALLY_URL/api/predicates

4. Create a rubric and start a run

# Create rubric with the predicate
curl -X POST \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -H "X-Project-Id: $PROJECT_ID" \
     -d '{
       "name": "Clarity Check",
       "predicates": [{"predicateId": "PREDICATE_ID", "weight": 1.0}]
     }' \
     $TALLY_URL/api/rubrics

# Start the evaluation run
curl -X POST \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -H "X-Project-Id: $PROJECT_ID" \
     -d '{
       "rubricId": "RUBRIC_ID",
       "corpusId": "CORPUS_ID",
       "model": "your-model"
     }' \
     $TALLY_URL/api/runs

5. Poll for results

# Check run status
curl -H "Authorization: Bearer $TOKEN" \
     $TALLY_URL/api/runs/RUN_ID

# Get scored results when complete
curl -H "Authorization: Bearer $TOKEN" \
     $TALLY_URL/api/runs/RUN_ID/results

Error responses

All errors return JSON with an error field:

{"error": "Unauthorized"}

Status	Meaning
400	Bad request — missing or invalid parameters
401	Unauthorized — missing or invalid token
403	Forbidden — insufficient permissions
404	Not found
409	Conflict — resource already exists
500	Internal server error

Interactive API reference

Try out endpoints directly in the API Playground.