When agents act, who will observe the agents?

For a long time, observing a system meant tracking relatively familiar signals. Availability, latency, errors, resource consumption, logs, metrics, and traces help teams understand whether an application is working, where there is a degradation, and what happened before a failure.

Artificial intelligence agents add a different difficulty to this problem. They do not just process an input and return a predictable output. They can interpret a situation, query different sources, choose tools, run a variable sequence of steps, interact with other agents, and, depending on the autonomy they are given, perform actions on real systems.

In this environment, knowing that the agent is available is only the beginning.

It may be responding within the expected time and without any infrastructure error, yet choosing an inadequate tool, repeating unnecessary calls, following a path different from the expected one, or producing results that begin to lose quality. When several agents take part in the same process, the difficulty increases because a decision made at one point can change the behavior of other components further down the line.

That is why observability begins to take on a different role in the agentic AI journey. It is not just about knowing whether the agent is working, but about being able to understand what it did, why it took a given path, and what happened as a consequence.

A trace stops showing only systems and starts showing decisions

In traditional applications, tracing is used to reconstruct the path a request takes across different components. That capability remains necessary in agentic systems, but what needs to be reconstructed changes.

A single interaction can involve several calls to a model, queries to external sources, context retrieval, tool use, additional attempts, and communication with other agents. The final response is only the last point of that trajectory.

The work being developed by OpenTelemetry (OTel) helps show how this new layer is beginning to be structured. OpenTelemetry is an open source project of the Cloud Native Computing Foundation (CNCF) that establishes standards and tools to generate, collect, and export telemetry, such as traces, metrics, and logs, making it possible to observe systems consistently regardless of the technologies used [1].

The semantic conventions for generative AI make it possible to record information such as the model used, token consumption, and operations related to agents and tools. In an instrumented execution, the trace can represent the agent invocation and, below it, the calls to the model and to the tools used during the process [1][2].

This changes the investigative capability quite a bit.

If an agent took too long to complete a task, we may need to find out whether the problem occurred in a model call, in an external API, or because it entered an unnecessary sequence of attempts. If it made an inadequate decision, it may be necessary to reconstruct which information it received, which tools were available, and which path it followed to reach that action.

Observing an agent is not just knowing whether it responded. It is being able to reconstruct the path that turned context into decision and decision into action.

This kind of visibility becomes even more important because agents do not behave exactly like deterministic software. Microsoft draws attention to this difference by noting that the same prompt can lead to different tool routes as models, prompts, and other conditions change. For that reason, its observability approach for agents combines tracing, evaluation, monitoring, and optimization, following everything from model and tool calls to interactions between agents [3].

The consequence is that traces, metrics, and logs remain fundamental, but now tell only part of the story.

An agent can be working and still be wrong

This may be one of the most important differences between observing traditional applications and observing agent-based systems.

An API that responds correctly within its technical parameters normally offers relatively objective signals that it is working. For an agent, technical health and quality of behavior are not necessarily the same thing.

It can remain available, show good latency, and consume resources as expected while its ability to complete a given task deteriorates. It can select incorrect tools more often, produce seemingly plausible answers that do not solve the problem, or start using more expensive paths to reach the same result.

That is why observability and evaluation begin to come closer together.

OpenTelemetry highlights that, because of the non-deterministic nature of agents, telemetry serves not only for diagnosis. It can also feed evaluation and continuous improvement mechanisms [1]. Microsoft follows a similar direction by combining execution traces with evaluations of quality, safety, and task completion [3].

In agentic systems, being available does not necessarily mean behaving correctly.

This difference has important practical implications. A mature operation will need to track technical signals, but also criteria tied to the outcome. Did the agent complete the task? Did it use the right tools? Did it respect the defined policies? Did the action produce the expected effect? Was human intervention needed? Is the behavior changing over time?

Not all of these questions will be answered by traditional metrics.

There will also be no single universal metric for all agents. A customer service agent will have different quality criteria from an agent that investigates incidents, just as an agent that only recommends an action may be evaluated differently from one authorized to execute it.

Observability will therefore also need to carry the context of what that agent was supposed to do.

When several agents work together, the problem stops being individual

The difficulty grows again when we move from one agent to multi-agent systems.

Imagine that a first agent receives an event, another gathers additional information, a third assesses possible causes, and a fourth executes or recommends an action. If the final result is wrong, at which point did the problem occur?

Perhaps no agent failed in isolation.

The first may have interpreted the event correctly but passed on incomplete context. The second may have found valid but insufficient information. The third may have made a reasonable decision based on what it received, while the fourth executed exactly the action requested. Each component, observed separately, may look correct even though the system’s result is wrong.

This characteristic led IBM researchers to argue that agentic systems can exhibit behaviors that are not understood by analyzing individual agents alone. In work presented at AAAI 2026, the international conference promoted by the Association for the Advancement of Artificial Intelligence and one of the main academic references in the field of artificial intelligence, the authors defend the importance of observability layers capable of tracking both the interactions between agents and the relationships between agents and the environment in which they act [4].

The more intelligence is distributed among agents, the more important it will be to observe the relationships between them, and not just each agent in isolation.

This shift recalls a known problem in distributed systems. For many years, monitoring servers or applications individually stopped being enough because the relevant behavior emerged from the dependencies between components. It was necessary to develop better ways to reconstruct transactions, relationships, and causes across the environment.

With agents, something similar may happen, but now there is an additional layer: beyond technical dependencies, we need to understand context transfers, decisions, delegations, and actions.

It is not enough to know that agent A called agent B. In certain situations, it will be necessary to understand what was transferred between them and how that influenced what happened next.

Observability will also be a form of governance

When agents only recommend actions, an inadequate decision can be noticed before it produces consequences. As they receive permission to execute tasks, the gap between decision and impact shrinks.

An agent can open a ticket, query corporate data, change a configuration, start a workflow, trigger another system, or request the action of another agent. The greater the autonomy, the greater the importance of keeping a trail that makes it possible to reconstruct what happened.

That trail does not serve only for troubleshooting. It becomes part of governance itself.

It will be necessary to know which agent performed a given action, under which version, with which permissions, using which information and tools, and, when applicable, who authorized the execution. Depending on the process, it will also be important to record when there was human intervention or when the agent decided to hand the situation over to a person.

IBM describes AI observability as the ability to understand AI models and AI-based systems from specific telemetry, including elements such as token usage, response quality, and model drift [5]. In agentic environments, this visibility extends to the agents’ interactions with APIs, services, and other components of the process.

As agents gain autonomy, observability stops being only a diagnostic tool and also becomes a mechanism of control and accountability.

This does not mean indiscriminately recording everything that passes through an agent. Prompts, responses, tool arguments, and data used during an execution may contain sensitive information. OpenTelemetry’s own documentation provides that certain content be recorded only when that capture is enabled [2].

Observability also needs to be governed.

The organization will have to decide what to record, how long to store it, who can query it, and how to protect that telemetry. The goal is not to create an unrestricted reproduction of everything the agents saw, but to ensure enough evidence to understand and govern their behavior.

Observing agents will be part of the infrastructure of trust

In the previous articles in this series, we discussed the move of agents into production.

The need for AgentOps also became clear on that journey.

We also saw the redesign of processes so that people, systems, automations, and agents can work together.

Observability connects these themes.

It is not possible to properly operate what we cannot understand. It is also hard to increase an agent’s autonomy when we are unable to reconstruct its behavior after something unexpected happens.

That is why agent observability should not emerge only as a reaction to the first incidents. It needs to be part of the architecture from the moment these systems begin to take part in relevant processes.

This includes instrumentation, traces, metrics, and logs, but also quality evaluation, tracking of actions, identification of agents, relationships between components, and the ability to connect what the agent did with what happened in the environment.

The discussion does not eliminate the observability we already know. On the contrary, it makes it even more necessary.

An agent does not act in a vacuum. It depends on models, APIs, databases, applications, networks, services, and information produced by other systems. When an action fails, it may be necessary to understand both the behavior of the agent and the state of the environment in which it was operating.

The trust to expand the autonomy of agents will depend, to a large extent, on our ability to see what they are doing when nobody is looking directly at them.

Perhaps this is one of the most relevant changes agentic AI brings to observability. Until now, we mainly observed systems to help people decide. In an environment with agents, we will need to observe also the intelligences that begin to take part in those decisions.

From vision to practice: ARGUS

When AI agents begin to carry out activities that matter to the operation, observing them requires understanding more than their internal functioning. It is necessary to track the tools they used, the decisions they made, the errors they found, the resources they consumed, and the results they produced.

This is one of the dimensions of the AgentOps proposal of ARGUS, which seeks to offer visibility and governance mechanisms over the actions of agents. But there is a second, equally important dimension: understanding the environment in which those actions took place.

With its AIOps capabilities, ARGUS correlates information from infrastructure, applications, services, and operational tools. Connecting these two perspectives makes it possible to relate the behavior of agents to the context of the operation, offering a more complete basis for investigation, supervision, and audit.

The difference is relevant: identifying that an agent executed a given action does not necessarily mean understanding why it happened or what its consequences were. For that, we need to observe both the agent and the operation in which it acts.

Learn about ARGUS and see how it works in practice.

About the author

Paulo Florêncio de Menezes is an entrepreneur, partner, and co-founder of companies such as Target Solutions, BiPTT, and ARGUS.

He holds a degree in Data Processing from the Federal University of Amazonas (UFAM), an MBA in Business Management from Fundação Dom Cabral (FDC), and a Professional Master’s in Economics from Fundação Getulio Vargas (FGV).

In his Master’s research, he developed the IBEOP-IA (Brazilian Index of Potential Occupational Exposure to Artificial Intelligence), investigating the potential exposure of Brazilian occupations to AI capabilities by combining occupational data, artificial intelligence patents, and natural language processing techniques.

References

[1] OpenTelemetry. AI Agent Observability – Evolving Standards and Best Practices. 2025.

[2] OpenTelemetry. Inside the LLM Call: GenAI Observability with OpenTelemetry. 2026.

[3] Microsoft. From observability to ROI for AI agents on any framework. Microsoft Build 2026.

[4] IBM Research. Formalizing Observability in Agentic AI Systems. AAAI 2026.

[5] IBM. What is AI Observability? 2026.