LLMOps and observability: AI tools compared (18)

What lLMOps and observability tools are for

LLMOps and observability tools help teams inspect applications that use language models or agents. They support people who need to understand an answer, compare changes and investigate problems after release. Depending on the service, you can review calls, follow execution steps and assess outputs against defined criteria. Previously, when an assistant produced an unsuitable answer, a developer might search scattered logs and try to reproduce the conversation. With a usable trace, the developer follows recorded steps and inspects the available evidence to narrow down the problem. Langfuse, for example, supports tracing agent calls and steps, measuring model usage and evaluating answers. These tools make investigation and comparison more organized. They still require suitable instrumentation and evaluation criteria tied to the application’s purpose. The recorded evidence also needs enough context to support a useful diagnosis without collecting unnecessary information.

Example uses:

• You inspect the trace behind an incorrect answer to identify the step that needs investigation or correction.
• You compare prompt versions against a representative test dataset before changing the behaviour of your deployed application.
• You monitor model calls to investigate unusual resource consumption following the release of a change to your application.
• You collect problematic examples into a test dataset and check whether proposed corrections address previously observed failures.
• You compare automated assessments with human feedback to refine the quality criteria used to evaluate your assistant.

Our picks by need

These picks follow simple rules applied to the verified data, with no partnership or affiliate link:

• Best scored : Langfuse. 95/100 on our access and transparency grid.
• Cheapest paid option : Evaligo. First plan at $7 per month.
• To keep data in Europe : Arize Phoenix. Processing on your device.

The main tools at a glance

Langfuse 95/100

Langfuse tracks agent calls and steps, measures model costs and usage, and evaluates responses. It serves people who want to manage instruction versions and compare trials on datasets. Features: Track an agent's calls and steps; Measure model costs and usage; Evaluate response quality. Entry price: $29 per month, with a free tier. Runs on: web browser and API. Data hosting: Several regions.

LangSmith 85/100

LangSmith examines agent steps and compares results across versions. It serves people who want to create test datasets and monitor the costs and response times of their agents. Features: Examine the steps followed by an agent; Compare results from multiple versions; Create test datasets. Entry price: $39 per month, with a free tier. Runs on: web browser and API. Data hosting: Several regions.

Opik 83/100

Opik lets teams developing applications that use models record execution steps and compare experiments. It evaluates responses using automated criteria, monitors production errors and tracks model call costs. Features: Record the steps of an application using models; Compare experiment results; Evaluate responses using automated criteria. Entry price: $19 per month, with a free tier. Runs on: web browser and API. Data hosting: Several regions.

Arize Phoenix 81/100

Arize Phoenix lets teams developing agents examine execution steps and evaluate response quality. It also supports comparing prompt versions and building datasets to use when testing their systems. Features: Examine an agent’s execution steps; Evaluate response quality; Compare multiple versions of a prompt. Free tier available. Runs on: web browser and API. Data hosting: On device.

Overmind 78/100

Overmind supports teams evaluating and training specialized models. It lets users inspect agent execution traces, prepare training data, assess model responses and deploy models behind an API. Features: Inspect agent execution traces; Prepare training data; Evaluate model responses. Entry price: $120 per month, with a free tier. Runs on: web browser and API. Data hosting: European Union.

MLflow 78/100

MLflow helps teams developing models and LLM applications compare experiment results. It records and versions models, tracks agent and application calls, evaluates responses and supports model deployment. Features: Compare experiment results; Record and version models; Track calls from agents and LLM applications. Free tier available. Runs on: web browser and API. Data hosting: On device.

Respan 71/100

Respan lets teams using models centralize their calls and track agent execution. It measures costs, errors, and latency, evaluates responses on test datasets, and versions model instructions. Features: Centralize calls to multiple model providers; Track agent execution steps; Measure costs, errors, and latency. Entry price: $199 per month, with a free tier. Runs on: web browser and API. Data hosting: United States.

Prisma AIRS AI Gateway — Portkey 71/100

Prisma AIRS AI Gateway — Portkey lets teams route requests between multiple models. It tracks costs and response times, applies rules, and manages access and budgets for each team. Features: Route requests between multiple models; Track costs and response times; Apply rules to requests and responses. Entry price: $49 per month, with a free tier. Runs on: web browser and API.

Lunary 68/100

Lunary helps teams using models track calls, responses, costs, latency, and errors. It manages instruction versions, supports response evaluation through human feedback, and masks personal data. Features: Record model calls and responses; Track costs, latency, and errors; Manage instruction versions. Entry price: $20 per month, with a free tier. Runs on: web browser and API. Data hosting: European Union.

TruLens 68/100

TruLens records agent steps for people evaluating their results. It assesses response relevance, measures latency and costs, and allows users to compare experiment results and view evaluations in a dashboard. Features: Record an agent's steps; Evaluate response relevance; Measure latency and costs. Free tier available. Runs on: web browser.

The category in figures

Of the 18 tools in the category, 18 (100%) offer a free tier, 100% have a public API and 0 have a French interface. The median entry price, per month, is $49 (across 13 tools with a published price). 9 disclose where data is hosted, 5 of them in Europe or on device.

Also useful

These tools belong to another category but also cover this need:

• Agenta Agenta is an open-source tool for people building agents with instructions and tools. It supports scheduled ta
• Mastra Mastra lets developers create agents connected to models and tools. It supports multistep workflows, conversat
• LiteLLM LiteLLM lets developers call multiple model providers through a common API. It distributes requests, switches
• NeuralTrust NeuralTrust lets teams using agents and models control access, requests and spending. It detects attempts to m
• Dynamiq Dynamiq lets teams build AI agents in a visual editor and connect them to company documents, applications, and
• Lakera Lakera helps people configure security rules for exchanges with models. It detects instruction hijacking attem
• Artificial Analysis Artificial Analysis lets developers compare AI model performance, provider pricing and speed. It offers model
• DataRobot DataRobot supports people training predictive models and building agents connected to data. It allows model co
• Manifest Manifest lets developers bring multiple model providers behind one API. It supports model configuration, provi
• Scale AI Scale AI prepares data and arranges dataset annotation for teams training models. It evaluates model performan
• Microsoft Foundry Microsoft Foundry lets teams test and deploy AI models and create agents connected to company tools. It also s
• Credyt Credyt lets teams billing for model usage measure costs by customer and charge consumption in real time. It ma

Choosing well and avoiding pitfalls

Try instrumentation in your actual application and check which steps become visible. A trace should connect an answer to the relevant calls and versions. Review integrations, event exports and experiment comparison features. For evaluations, test French material where relevant and assess whether criteria reflect your business expectations. Inspect masking, access controls and retention settings. Compare billing conditions with expected trace activity, evaluation runs and team access. Include the effort required to maintain instrumentation. The collected evidence should help answer practical debugging questions rather than simply accumulate records that nobody can interpret.

Traces can capture confidential messages, documents and tool results. Filter sensitive material before transmission and restrict access. Automated evaluators may favour convincing language over factual accuracy. Compare their judgments with examples reviewed by qualified people. Aggregate results can hide consequential failures, so inspect individual cases. Check whether collection misses the errors you need to investigate. Also monitor the resources used by tracing and evaluation, and make sure the evidence remains interpretable when execution paths or application versions change over time.

To compare with the cost of using an AI model directly, see the API price calculator.

Free or paid?

Free plans may let you instrument a prototype and explore traces within activity or retention restrictions. They are useful for checking whether collected information supports investigation. Paying can become relevant for production monitoring, shared debugging, longer history or access controls suited to your organization. Review evaluation usage separately if it is governed by different conditions.

Tools in the category with a free tier: Langfuse, LangSmith, Opik, Arize Phoenix, Overmind, MLflow, Respan, and Prisma AIRS AI Gateway — Portkey.

For which jobs?

The profiles that use these tools most, based on the use cases documented by publishers: developers, data analysts, researchers, project managers, and customer support.