Skip to content

Quickstart

Get checkllm running in under 5 minutes.

Install

pip install checkllm

Initialize

checkllm init --use-case rag

This creates a tailored test file, detects your API keys, and configures pyproject.toml.

Write Your First Test

def test_output_quality(check):
    output = my_llm("What is Python?")

    # Free, instant checks (no API key needed)
    check.contains(output, "programming language")
    check.max_tokens(output, limit=200)
    check.no_pii(output)

    # LLM-as-judge checks (auto-detects your API key)
    check.hallucination(output, context="Python is a programming language.")
    check.relevance(output, query="What is Python?")

Run

pytest tests/ -v

Estimate Costs Before Running

checkllm estimate tests/

Track Regressions

checkllm snapshot tests/ --output baseline.json
# ... make changes ...
checkllm snapshot tests/ --output current.json
checkllm diff --baseline baseline.json --current current.json

Next Steps