Fiddler AI

Editor’s note: This Fiddler AI review is based on my review of the Fiddler platform (release 26.25) and its Python client 3.14.1 in October 2026. I weigh pricing, setup effort, agent and ML monitoring depth, guardrails, and deployment options.

Quick verdict: I recommend Fiddler AI for regulated enterprises (banking, insurance, government) that run predictive models and LLM agents side by side and need guardrails inside their own environment. The catch: past Developer’s $0.002 per trace, you’re in a sales conversation (AWS Lite alone is a $24,000 12-month contract).

Key Takeaways

• Three products in one: ML monitoring with explainability, agent tracing, and Centor guardrails.
• Free means guardrails, not tracing: observability starts on Developer at $0.002 per trace.
• Guardrails stay in your cloud: Centor Models run in your VPC or air-gapped, with latency quoted under 80ms.
• Enterprise is quote-only: AWS Lite costs $24,000 a year for 1 model and 0.5 GB of data a month.
• Rough edges: alerts, saved segments and custom metrics can’t be edited, and OTLP ingestion is HTTP only.

In this Fiddler AI review, I’ll take a closer look at Fiddler’s pricing, setup, features and rivals, so you can see exactly whether this LLM observability and evals platform, which still treats tabular models as seriously as agents, is worth an enterprise contract.

Fiddler AI Pros and Cons

Fiddler AI Pricing

Fiddler AI pricing has three public plans, and only Developer publishes a rate:

• Free ($0): for real-time guardrails against hallucinations, toxicity, PII/PHI leaks, prompt injection and jailbreaks, powered by Centor Models.
• Developer ($0.002 per trace): for teams that want observability across agentic and predictive systems, plus tests and experiments, custom evaluators, bring-your-own-judge, and RBAC with SSO. SaaS only.
• Enterprise (custom): for SaaS, VPC or on-prem deployment with enterprise guardrails, dedicated support, customized onboarding and a named Customer Success Manager.

Plan	Price	Deployment	Best for
Free	$0	Not stated (guardrails run in your environment)	Testing guardrails
Developer	$0.002 per trace	SaaS	Agent and predictive observability pilots
Enterprise	Custom quote	SaaS, VPC, on-prem	Regulated production deployments

The per-trace math is simple: 10,000 traces a month costs $20, 100,000 costs $200, and 1 million costs $2,000. That’s cheap for an agent pilot, but Fiddler publishes no price list beyond Developer, so a production predictive-monitoring budget means asking for a quote.

Other Ways to Buy

If your procurement runs through AWS, you have two more routes:

• AWS Marketplace Lite: a $24,000, 12-month contract for 1 model, 0.5 GB of data a month and the lowest support tier. The listing says no LLM functionality is included, yet its overview describes agent observability, so get the scope in writing before you sign.
• SageMaker Partner AI App: a 30-day free trial for up to 5 models, running inside your own AWS account, then monthly or annual billing. AWS bills the infrastructure separately.

Fiddler’s infrastructure estimates for the SageMaker app:

• Small: $500-$1,000 a month (under 10 users).
• Medium: $1,500-$3,000 a month (10-50 users).
• Large: $3,000-$6,000 a month (50+ users).

One gotcha: a trial install can’t be upgraded to paid without redeploying, so treat the trial as a sandbox, not your future production instance.

Is Fiddler AI Good Value for Money?

• Cheap tracing is cheaper elsewhere: Langfuse Core is $29 a month and Arize AX Pro is $50, both with flat, published plans.
• LangChain teams have a natural fit: LangSmith Plus costs $39 per seat a month if you don’t need predictive monitoring.
• Datadog shops pay per span: Datadog LLM Observability Pro is $160 a month for 100,000 LLM spans, and spans don’t map one-to-one to Fiddler traces.
• Fiddler’s edge is private evaluation: Centor evaluators run in your environment with no per-evaluation LLM API charge, which matters more as your volume grows.

Reviewer’s Notes: Start on Free if guardrails are the job, and move to Developer for a SaaS agent pilot. I’d only ask for Enterprise pricing once you need VPC or on-prem deployment, or ML monitoring at production volume.

My Experience With Fiddler AI

Fiddler isn’t a log-in-and-look tool. Setup is code-first: you define a schema for each predictive model and instrument each agent before a single dashboard fills up.

Onboarding a Predictive Model

Everything starts with the Python client: pip install fiddler-client, which needs Python 3.10 or newer. The current version is 3.14.1, but keep an eye on the compatibility matrix. Client 3.14 maps to platform releases 26.19-26.22, and 26.23 already needs a newer client, so pin both versions together.

From there, onboarding a model takes five steps:

1. Create a project with fdl.Project(name=...) and project.create(). Names must be unique, with no spaces or special characters.
2. Define a ModelSpec that assigns each column to inputs, outputs, targets, decisions, metadata or custom features.
3. Set the task (optional), for example binary classification with its target class order.
4. Infer the schema with fdl.Model.from_data(...), passing your event ID and timestamp columns, review the inferred types, then onboard with model.create().
5. Publish data: production events plus an optional baseline dataset. Fiddler then generates metrics, anomaly flags and charts automatically, and you add dashboards and alerts on top.

Slow down at step 4. The ModelSpec decides how Fiddler reads your predictions and outcomes, and a baseline you pick on purpose gives drift monitoring a reference you actually trust.

Instrumenting an Agent

Agent tracing is quicker. You create a project and an application, then set three environment variables:

• FIDDLER_APPLICATION_ID: the application receiving traces.
• FIDDLER_API_KEY: authentication.
• FIDDLER_URL: your Fiddler instance.

LangGraph and LangChain apps use LangGraphInstrumentor from fiddler_langgraph, and Strands has its own SDK. Anything else goes through OpenTelemetry, with an OTLPSpanExporter behind a BatchSpanProcessor. OTLP must go over HTTP, so a gRPC collector config won’t work as-is. Fiddler says traces show up within a minute or two.

Traces land in a hierarchy (application, session, agent, trace, span) next to a timeline. The hierarchy tells you which agent misbehaved, and the timeline shows the order it did things in.

Adding Guardrails

Guardrails take the least work of the three. Generate an API key under Settings > Credentials, keep it in your secrets manager, and call the /v3/guardrails/* REST endpoints. Then pick the checks you need: Safety, Faithfulness or PII/PHI, with secrets covered by the Sensitive Information guardrail.

Reviewer’s Notes: Hand setup to an ML or platform engineer, not a business analyst. ModelSpec design, OTLP transport and client pinning all need someone who knows the model’s inputs and your telemetry pipeline.

Agentic Observability and Evaluations

Plenty of LLM tracing tools show you a flat list of calls. Fiddler LLM observability organizes them into five levels: application, session, agent, trace and span, which is exactly what you want once a supervisor starts delegating. Fiddler’s demo uses a supervisor with hotel and travel sub-agents: you follow one request down to the sub-agent that went wrong, then into its span.

Evaluations hook into the same structure:

• Evaluator Rules: map checks to span attributes.
• Continuous Evaluations: keep scoring production traffic.
• Custom evaluators and bring-your-own-judge: included in Developer, so you can score booking correctness, not just generic safety.
• Tests and experiments: compare an agent change before launch instead of trusting a handful of good traces.

Integration Coverage

• LangGraph and LangChain: through fiddler_langgraph.
• Strands: a dedicated Python SDK.
• Google ADK: a supported integration.
• OpenTelemetry: OTLP over HTTP for everything else.
• Claude Code: a plugin in public preview, with a Windows preview added in 26.23.
• Gateways: Portkey, Kong and LiteLLM.

Release 26.25 added LLM Gateway models and REST/MCP editing of eval dataset items with versioning. That versioning tells you whether a score moved because the agent improved or because someone edited the test set. The same release added an Evaluator Cost API field in private preview, so you can see what each evaluator costs to run, on top of the 80+ out-of-the-box evaluators Fiddler ships.

The limits are real: the LangGraph and Strands SDKs are Python-only, gRPC ingestion is still only planned, and attribute limits apply to each span, so very chatty agents may need trimming. Team admins got a little help in 26.24, which made the Teams and Project Roles REST APIs public and put service accounts in private preview. If you build on LangChain alone, LangSmith fits more naturally, and if you want free self-hosted tracing, start with Langfuse. Fiddler earns its place when one purchase has to cover predictive models and private guardrails.

Guardrails and Centor Models

Most observability tools tell you something went wrong after a user already saw it. Fiddler’s Centor Models block inline, inside your VPC or an air-gapped environment, with no external LLM API calls. That’s the strongest reason I’d pick Fiddler over a tracing-only tool.

Centor Models (formerly Trust Models) are small, task-specific models, one per check:

• Faithfulness: flags hallucinated or unsupported output.
• Safety: catches jailbreaks and toxicity categories such as harassing, harmful, hateful, illegal, sexual and violent content.
• Sensitive Information: detects PII across 35+ entity types, healthcare data and secrets. Release 26.24 added heuristic validation and secret detection to this check.

Latency is quoted as under 80ms on the pricing page but sub-100ms in the guardrails docs, and Fiddler says it handles more than 5 million requests a day. Plan around the 100ms figure until you’ve timed it on your own traffic.

The Free plan includes these checks, and because they run locally there’s no per-evaluation token bill. You still pay for the infrastructure they run on, and if a Centor check doesn’t match your policy, bring-your-own-judge covers the gap.

Check Dynamic Model Escalation before you approve a strict data boundary. This optional fallback, in private preview for Centor Safety, sends out-of-distribution inputs to a Fiddler-hosted general-purpose model, so keep it off if data can’t leave your network.

Fiddler also claims 99% precision on jailbreaks and up to 98% lower evaluation cost, with no published methodology for either. Run your own red-team prompts, plus legitimate requests that shouldn’t be blocked, before you trust those numbers.

The closest rival here is Galileo, now Splunk Agent Observability, whose Luna eval models can drive guardrails in a similar loop.

ML Monitoring, Explainability and Fairness

LLM-first tools rarely handle a credit-scoring model well. Fiddler AI model monitoring still does: drift, data integrity and performance tracking for predictive models, with Feature Impact for explainability and an Executive Dashboard for the people who sign off on risk.

Alerts and Cohort Monitoring

You get six alert types:

• Traffic: production event volume.
• Statistics: changes in statistical values.
• Data Drift: distribution shifts.
• Data Integrity: data-quality problems.
• Performance: predictive performance.
• Custom Metric: any aggregate you define.

Each rule uses an absolute or relative comparison, a required Critical threshold, an optional Warning threshold, and Low, Medium or High priority. Notifications go to email or PagerDuty (once an admin enables it), and a Datadog integration sits in the integrations list.

Segments are boolean, row-level Fiddler Query Language expressions, so you can check whether a performance dip hits one customer cohort or the whole model. You can also define segments on the fly in Analytics or Monitoring charts and set alerts on them.

Getting data in is the easy part, with integrations for Snowflake, BigQuery, Kafka, S3, Airflow, SageMaker, Databricks and MLflow.

Custom Metrics and Fairness

FQL uses Excel-like syntax over features, metadata, predictions and outcomes, and every custom metric must return an aggregate. Functions include quantile() and gini(), though gini() only works for ML, not GenAI or agent apps.

Fairness is built from custom metrics:

• Disparate impact: the four-fifths rule.
• Group benefit: benefit rates across groups.
• Equal opportunity: outcome gaps between cohorts.
• Demographic parity: prediction rates across groups.

EEOC protected attributes and intersectional subgroups work too, but this is a toolkit, not a certification: your risk team still picks the metrics and the cohorts.

My biggest complaint is that saved segments, custom metrics and alert rules can’t be edited. Change a threshold, and you delete and recreate the rule.

If you only need drift reports, Evidently AI’s open-source library is the lighter option. With WhyLabs discontinuing operations, Fiddler and Evidently are the obvious landing spots for its users.

Deployment, Security and Release Pace

Fiddler offers VPC and on-prem deployment, which gets it through security reviews that rule out SaaS-only tools:

• Developer: SaaS.
• Enterprise: SaaS, single-tenant SaaS, VPC or on-prem.
• Self-hosted: AWS, GCP or Azure, plus AWS GovCloud.
• Guardrails: VPC or air-gapped.

The security controls cover the usual procurement checklist:

• Compliance: SOC 2 Type II (report under NDA) and HIPAA.
• Encryption: TLS 1.2+ in transit, AES-256 at rest, daily encrypted backups.
• Identity: RBAC plus SSO for Google, Microsoft Entra ID, Okta and PingOne.
• Networking: AWS VPC Endpoint and PrivateLink.

Fiddler doesn’t state FedRAMP authorization, and GovCloud availability isn’t the same thing, so government buyers should ask directly.

If you go the SageMaker route, deployment runs through a six-step wizard in 1-2 hours, but the execution role name and admin usernames can’t be changed afterwards, so choose them carefully.

The company raised a $30M Series C in January 2026 ($100M in total) and bought coding-agent security firm Lumeus in April. Releases 26.20 through 26.25 shipped in about five weeks, so pin your client version and test each upgrade in staging.

How Fiddler AI Compares to Competitors

Tool	Starting price	Self-host/VPC	Wins for
Fiddler AI	Free guardrails; Developer $0.002/trace	Enterprise	Mixed ML and agents with private guardrails
Arize AX	Free; Pro $50/month	Enterprise; Phoenix is local-first	Low-cost ML and LLM observability
Langfuse	Free; Core $29/month	Free self-hosting	Open-source LLM tracing
LangSmith	Free; Plus $39/seat/month	Enterprise	LangChain-native development
Datadog LLM Observability	Free; Pro $160/month	SaaS	Existing Datadog environments
Arthur AI	Not published	Not published	Agent security and CISO workflows

Don’t compare these entry prices line for line: traces, spans and Langfuse units are different billing units, and Fiddler’s Free tier covers guardrails, not tracing.

• Arize AX: the cheaper way into observability, with 25,000 free spans a month (1 GB ingestion, 15-day retention) and Pro at $50. Self-hosting, SOC 2 Type II and HIPAA are Enterprise-only, though. Dynatrace agreed to acquire Arize for $915 million in August 2026, with closing pending.
• Langfuse: my first pick for free self-hosting. Cloud plans cost $29, $199 and $2,499 a month, but units count traces, observations and scores, and SSO/RBAC is a $300-a-month add-on.
• LangSmith: the natural fit for LangChain builders at $39 per seat on Plus. Base traces are kept 14 days, and self-hosted or hybrid deployment needs Enterprise.
• Datadog LLM Observability: the default for Datadog shops, with 40,000 free LLM spans a month, Pro at $160 for 100,000, and traces tied into APM, but 15-day retention on the listed tiers.
• Arthur AI: better for CISO-led buying, where shadow-agent discovery and SIEM streaming matter more than model explainability. It also offers runtime guardrails and spend tracking.
• Galileo: now Splunk Agent Observability, worth a look for Luna-based evals and guardrails, especially if you already buy from Splunk or Cisco.
• W&B Weave: the easy choice when your experiment tracking already lives in W&B.

For open-source drift and evaluation, add Evidently AI to the list: an Apache 2.0 library with 100+ metrics plus a self-hosted platform. Fiddler wins when private guardrails and predictive-model governance have to live in one platform.

How I Reviewed Fiddler AI

I evaluated Fiddler’s onboarding path for two workloads: a predictive model (client install, ModelSpec, baseline and production events) and an agent (application setup, LangGraph and OpenTelemetry instrumentation). I also walked through guardrail setup over the REST API.

For pricing, I worked out Developer costs at 10,000, 100,000 and 1 million traces a month, and compared price, deployment and guardrails against Arize AX, Langfuse, LangSmith, Datadog LLM Observability and Arthur AI. Finally, I checked Fiddler’s compliance claims, its release history from 26.20 to 26.25, and how its guardrail accuracy claims stack up against what it actually publishes.

Prices current as of October 2026.

Should You Use Fiddler AI?

Yes, if predictive monitoring and agent protection belong in the same purchase. Explainability, cohort fairness metrics and in-environment Centor guardrails make a stronger case than the tracing price alone.

Recommended if:

• You work in a regulated industry: banking, insurance or government teams with strict deployment rules get VPC, on-prem and GovCloud options.
• You run a mixed AI portfolio: predictive models and LLM agents share one platform.
• You need inline protection: Centor guardrails block content inside your VPC or air-gapped.

Not recommended if:

• You just want cheap LLM tracing: Langfuse or LangSmith give you flat, published plans with far less setup.
• You’re standardized on Datadog: Datadog LLM Observability ties into the APM you already pay for.
• You need proven guardrail accuracy up front: Fiddler publishes no precision or recall methodology.

Your best next step is the Free guardrails plan or the SageMaker 30-day trial, before you book an Enterprise sales call.

Fiddler AI FAQ

Is Fiddler AI free?

Partly. The Free plan covers real-time guardrails, not observability. Tracing starts on Developer at $0.002 per trace, so 100,000 traces cost $200 a month. For predictive monitoring, the SageMaker Partner AI App offers a 30-day trial for up to 5 models, with AWS infrastructure billed separately.

Is Fiddler AI open source?

No. Fiddler is a commercial platform, and its Python client is published on PyPI for connecting your models and agents. If you need open source, look at Langfuse for LLM tracing or Evidently AI for drift and evaluation.

Can Fiddler AI run on-premises?

Yes. Enterprise covers on-prem and VPC deployment, single-tenant SaaS, self-hosting in AWS, GCP or Azure, and AWS GovCloud. Centor guardrails can also run air-gapped. Keep Dynamic Model Escalation off if no data may leave your network.

Is Fiddler AI SOC 2 and HIPAA compliant?

Yes. Fiddler states SOC 2 Type II and HIPAA compliance, and shares the SOC 2 report under NDA, so request it during procurement. FedRAMP authorization isn’t stated, and GovCloud availability doesn’t establish it.