Detecting is easy. Finding the root cause takes context

In the article “Your operation monitors everything. So why does it still get so many alerts?”, I shared a reflection on a problem that has become increasingly common in modern IT operations. As we expanded monitoring coverage, we began to see far more signals about how environments behave, but that did not necessarily mean a faster understanding of incidents.

A single problem can produce dozens or even hundreds of alerts spread across different tools. Each platform correctly records what it can observe and does very well the job it was designed for. The challenge begins when the operation needs to turn that set of information into a coherent explanation of what is really happening.

That is precisely when an important question arises.

If today we can detect an incident in a few seconds, why do we still take so long to find out where it began?

This question deserves attention because, in many environments, the slowest stage is no longer noticing that a problem exists. The real challenge has become understanding its origin before the impact on the business grows.

Detecting is only the beginning

Imagine a relatively common situation in a large company.

It is two in the morning and a critical application begins to slow down. Within seconds, monitoring records an increase in CPU usage on some servers. The observability platform identifies growth in application response time. The digital experience solution detects degradation in the user journey. The ITSM system starts receiving tickets and, shortly after, other applications begin to show similar behavior, even though they are hosted in different environments.

In less than a minute, the operation already knows there is a significant incident.

The curious thing is that, at this moment, nobody knows exactly what the problem is.

The infrastructure team quickly checks server utilization and finds no abnormality that could explain the observed behavior. The database keeps responding within the expected parameters. The network appears stable. Cloud services report no outages. Each team begins analyzing the components under its responsibility while new information keeps arriving.

The pressure rises. Managers want an estimate of when things will return to normal. Business areas ask which services were affected. Users keep opening tickets. Meanwhile, the technical teams are still trying to answer a much more basic question: what actually happened?

This situation is far more common than it seems.

For many years, we believed that expanding monitoring would automatically reduce the time needed to resolve incidents. In a way, that did happen. Today we detect problems much faster than in the past.

But environments have changed too.

Applications no longer depend only on physical servers and now run on distributed infrastructure, cloud services, APIs, microservices, replicated databases, messaging platforms, and many shared components. At the same time, new specialized tools appeared to monitor each of these layers.

This evolution brought important gains in scalability, visibility, and reliability. In return, it significantly increased the amount of information produced during an incident.

If an operation used to analyze few event sources, today it has to deal simultaneously with infrastructure monitoring platforms, observability, APM, centralized logs, network monitoring, digital experience, cloud, containers, Kubernetes, security tools, ITSM, CMDB, and various vendor-specific solutions.

Each of them observes the environment from a different perspective.

Each produces events, metrics, logs, indicators, and alerts that make sense within its own context.

Each correctly describes what it can see.

The problem is that none of them was designed to explain, on its own, an incident that crosses all these layers.

That is exactly where the complexity of modern operations begins to show.

Before finding the cause, you need to build context

Let’s go back to the incident.

After almost forty minutes of investigation, someone notices that all the affected applications share the same authentication service. A few minutes before the problems began, a new version of that component had been deployed.

From that moment, several events that seemed independent begin to make sense.

The authentication errors stop being seen as an isolated problem. The slowness observed in other applications comes to be interpreted as a consequence of the same failure. The tickets logged by the Service Desk show exactly the same start time. The time sequence of events begins to indicate a clear relationship between the update and the symptoms users perceived.

Notice what happened.

The operation did not immediately find the root cause.

Before that, it managed to reconstruct the story of the incident.

That difference seems small, but it helps explain why so many investigations take so long.

In practice, reconstructing that story means much more than gathering alerts on a single screen. It means understanding how information produced by different platforms relates to each other and how each one represents only part of reality.

An increase in CPU usage may be just the consequence of an earlier failure. An error logged in the application may originate in a shared service used by dozens of other systems. A degradation perceived by users may have started several minutes after the change that really triggered the incident. A large volume of tickets may be just the reflection of a problem that had been silently spreading through the environment.

None of this information is enough, on its own, to explain what happened.

It is precisely when this evidence starts to be correlated that the operation begins to build the context of the incident.

This context does not come from a single tool or a single data source. On the contrary. It depends on the ability to integrate information from multiple platforms and interpret it together.

Monitoring alerts need to be related to observability metrics. Logs need to be compared against recent changes made to the environment. ITSM information helps identify the impact users perceived. The topology reveals dependencies between applications. The history of similar incidents offers important references. The time sequence shows which events happened first and which appeared only as a consequence.

Separately, these elements represent only fragments of reality.

When analyzed together, they begin to build a coherent narrative about the behavior of the environment.

That is what we call operational context.

Context does not simply mean consolidating dashboards or centralizing events from different vendors. Nor does it mean only reducing the number of alerts shown to the operation.

Building context means correlating heterogeneous information until it explains the environment’s behavior. It means understanding which alerts are just symptoms, which belong to the same incident, which components share dependencies, which changes may have started the failure, and which evidence really helps steer the investigation toward its origin.

This may be one of the most important changes IT operations are going through.

For a long time, we believed that producing more data would be enough to make better decisions.

Today we realize that the real differentiator is not the amount of information available, but the ability to quickly turn it into understanding.

The differentiator of the most mature operations

There is a widespread idea that mature operations are those that monitor more assets, have more dashboards, or use more sophisticated tools.

Without a doubt, all of that is important.

But, having followed projects in different organizations over the past years, I came to notice that the real differentiator usually shows up at another moment.

It shows up in how quickly the operation can build context.

This may be one of the least discussed and, at the same time, one of the most important competencies of modern operations.

The more complex the environment becomes, the greater the number of specialized platforms tends to be and, consequently, the greater the number of sources producing information about the same incident. The problem is that each of them uses its own language, its own data models, and its own way of representing what is happening.

While one tool records events, another works with metrics. A third produces logs. Another presents distributed traces. ITSM records tickets. The CMDB describes relationships. Cloud platforms report infrastructure changes. Security solutions identify suspicious behavior.

Each of these pieces of information has value.

But none of them, on its own, explains an incident that crosses all these layers.

In practice, the operation’s work has become much less about finding data and much more about discovering how that data relates.

That is exactly why building context has become such a strategic capability.

When the operation can correlate information from multiple sources, it stops seeing dozens of independent events and begins to understand a single incident.

What used to look like a succession of disconnected problems begins to reveal cause-and-effect relationships. Alerts stop competing with each other. Symptoms are distinguished from the origin of the failure. Dependencies between applications help eliminate unlikely hypotheses. Recent changes become part of the investigation from the first minutes. The history of similar incidents offers references that would hardly be noticed by analyzing a single tool.

Notice that this transformation does not happen because more data appeared.

It happens because the data started to make sense together.

Perhaps that is the best definition of operational context.

Context is what turns data into understanding.

This understanding completely changes how teams work.

Instead of each specialist interpreting only the part of the environment they can see, the operation begins to share a common view of the incident. The conversation stops revolving around dozens of alarms and focuses on what really matters: what was the first deviation observed, how did it spread through the environment, which services were impacted, and what evidence points to the origin of the problem.

This does not eliminate the need for experienced professionals.

Much less does it mean the root cause will be identified automatically.

Critical environments will continue to require technical knowledge, analytical ability, and human judgment.

What changes is the starting point.

Instead of starting each investigation by trying to manually organize hundreds or thousands of scattered pieces of information, teams begin to work on a much clearer understanding of the scenario they are facing.

This difference produces effects that go far beyond identifying the root cause.

The right teams are mobilized earlier. Symptoms stop being treated as independent incidents. Communication between areas becomes more objective because everyone works from the same interpretation of the facts. The time spent discussing unlikely hypotheses decreases and the operation’s energy is directed toward what really needs to be resolved.

Perhaps this is precisely the most important transformation IT operations are going through.

For many years, our main goal was to expand the ability to detect failures.

Today that ability is practically a basic requirement.

What begins to set truly mature operations apart is how quickly they can build context, turning thousands of pieces of information produced by multiple sources into a consistent understanding of what really happened.

In the next article in the series, “Observability is not a tool. It is a new way of operating.”, we will deepen this discussion and analyze why observability goes far beyond collecting metrics, logs, and traces. Its real value lies in the ability to broaden the understanding of the relationships among applications, infrastructure, and services, enriching the operational context needed for decisions to be made much faster and with more confidence.

Food for thought

When a significant incident occurs in your operation, how much time is spent fixing the problem and how much is spent simply trying to understand where it began?

Can the different teams quickly build a common view of the incident, or does each one initially work with a different interpretation based only on the tool it uses?

Is the information produced by monitoring, observability, logs, ITSM, cloud platforms, and other solutions really correlated, or does it continue to be analyzed in isolation?

Are recent changes, dependencies between applications, infrastructure topology, history of similar occurrences, and business impact part of the investigation from the first minutes, or are they only considered after a long analysis?

Answering these questions may reveal that the biggest challenge of modern operations is no longer detecting incidents.

It is quickly building the context needed to understand what really happened.

About the author

Paulo Florêncio de Menezes is an entrepreneur, partner, and co-founder of companies such as Target Solutions, BiPTT, and ARGUS.

He holds a degree in Data Processing from the Federal University of Amazonas (UFAM), an MBA in Business Management from Fundação Dom Cabral (FDC), and a Professional Master’s in Economics from Fundação Getulio Vargas (FGV).

In his Master’s research, he developed the IBEOP-IA (Brazilian Index of Potential Occupational Exposure to Artificial Intelligence), investigating the potential exposure of Brazilian occupations to AI capabilities by combining occupational data, artificial intelligence patents, and natural language processing techniques.