MLflow Tracing
Observability
Open-source LLM and agent tracing inside MLflow: one-line autolog, OpenTelemetry ingest, LLM judges. Free to self-host under Apache 2.0.
LLM observability and evals
Traces, evals and monitoring for applications built on language models.
Loading tools…
HoneyHive
Observability
LLM and agent observability with OpenTelemetry tracing, online evals, CI regression gates, alerts and annotation queues.
Agenta
Observability
Open-source LLMOps and agent workspace: prompt versioning, 15+ evaluators and OpenTelemetry tracing, MIT-licensed and self-hostable.
Datadog LLM Observability
Observability
Datadog's LLM and agent observability: traces model calls and agent steps, runs LLM-judge evals and experiments, bills LLM spans only.
New Relic AI Monitoring
Observability
LLM and agent monitoring inside New Relic APM: model calls traced as spans beside database and service calls, billed as ordinary ingest.
Fiddler AI
Observability
Enterprise ML and LLM agent observability with in-environment guardrails, explainability and fairness metrics.
Free open source; Enterprise self-hosted by quote Evidently AI
Evals
Open-source Python library for LLM output evals with LLM judges, plus data drift and model quality reports and a self-hostable dashboard.
Maxim AI
Evals
Agent simulation, evaluation and observability platform that tests multi-turn chat and voice agents with simulated users, evaluators and tracing.
Laminar
Observability
Open-source observability for AI agents: transcript traces, Signals failure detection, debugger replay and evals, from $30/month.
Openlayer
Evals
AI testing, tracing and governance platform: 100+ catalog tests in CI and on live traces, guardrails, Gateway, EU AI Act mapping.
Confident AI (DeepEval)
Evals
LLM evaluation and observability platform built on open-source DeepEval: 50+ metrics for tests, CI and live production traces.
Free; Pro Cloud from $19/month
Opik (Comet)
Observability
Open-source LLM observability and evaluation from Comet, with agent tracing, datasets, online scoring, prompt versioning, and optimization.