Confident AI (DeepEval)
Evals
LLM evaluation and observability platform built on open-source DeepEval: 50+ metrics for tests, CI and live production traces.
LLM observability and evals
Traces, evals and monitoring for applications built on language models.
Loading tools…
Free; Pro Cloud from $19/month
Opik (Comet)
Observability
Open-source LLM observability and evaluation from Comet, with agent tracing, datasets, online scoring, prompt versioning, and optimization.
From $20/user/mo
Lunary
Observability
LLM observability platform linking agent traces, chatbot conversations, feedback, prompt templates, and evaluation workflows.
Free tier; Production from $49/month
Portkey
Gateway
AI gateway with provider routing, fallbacks, request logs, guardrails, and prompt management. Now branded Prisma AIRS AI Gateway.
Free tier; Enterprise quote
Traceloop
Observability
LLM tracing, evaluations, and prompt management built around the Apache-2.0 OpenLLMetry SDK. The commercial platform has a free hosted tier and quote-based Enterprise deployment.
Free open source; usage costs extra
TruLens
Evals
Open-source Python framework for evaluating RAG applications and AI agents with configurable metrics, OpenTelemetry traces, and version comparisons.
Promptfoo
Evals
Open-source CLI for LLM evals and red teaming: YAML test suites, 60+ providers, CI gates and 157 attack plugins, now owned by OpenAI.
Free open source; enterprise by email Ragas
Evals
Open-source Python library for scoring RAG and LLM apps with LLM-judged metrics and generating synthetic test questions from your documents.
Galileo
Evals
LLM and agent eval platform with 31 built-in metrics, Luna-2 judge models and production Signals, now sold as Splunk Agent Observability.
Free; Pro from $60/mo Weights & Biases Weave
Observability
LLM observability and evaluation toolkit for tracing AI applications, comparing outputs, and turning failures into repeatable tests.
Helicone
Observability
Proxy-based LLM observability gateway with one-line setup; hosted product in maintenance mode since the March 2026 Mintlify acquisition.
Arize AX
Observability
Enterprise LLM/ML observability (AX) with Phoenix, its ungated open-source tracing and evals core; deep drift and embeddings analysis.
12