Sharing & Data Model

Tally organizes work into projects that can be shared across team members. This page explains how data is structured and how collaboration works.

Projects

Projects are the top-level organizational unit in Tally. Each project contains its own corpora, rubrics, and evaluation runs. You can switch between projects using the project selector in the sidebar.

Projects enable collaboration — multiple team members can work within the same project, sharing access to the same content, rubrics, and runs.

Data Hierarchy

Project
├── Corpora (collections of documents)
│   └── Documents/Content (individual text items)
├── Rubrics (evaluation frameworks)
│   └── Predicates (individual scoring criteria)
└── Runs (batches of LLM executions)
    └── Evaluations (per-document scores)

Corpora & Documents

A corpus is a collection of text documents that you want to evaluate. Documents within a corpus are the individual units that get scored against your rubrics.

A document can be in multiple corpuses - a fairly commoon pattern is to create a "main" corpus, and then create a "test" corpus with a subset of the documents for quick iteration.

Rubrics & Predicates

A rubric defines what you're evaluating. Each rubric contains one or more predicates — specific criteria that produce a score. Predicates are designed through conversation and use log probabilities to produce calibrated scores.

Runs & Results

A run executes a rubric against a corpus, producing scores for each document-predicate pair. You can compare runs side by side to track how changes to rubrics or content affect outcomes.

Sharing

Projects can be shared with other users (with either READ or WRITE access), giving them access to all corpora, rubrics, and runs within that project. Sharing is managed through the project settings page.