Six experts, one verdict. SRE calls them in.

Network, infrastructure, application, logs, hosts and vulnerabilities: each expert reads its own tools, the orchestrator agent puts the findings together. All read-only.

Discover Agents

Orchestrator agent

Hypotheses · timeline · report

Logs: 14 timeout errors at 13:41 on the store's index. Lines attached.

Illustrative example · every request to the tools stays in the audit log

An investigation, from alert to report.

Nobody has to open six consoles one after another: the agent does it for the person on call, and reports what it found.

  1. 13:42

    The alert arrives.

    A message, for example on Teams, starts the investigation: it says what is not working and where.

  2. 13:42

    An isolated container opens.

    Each investigation works in its own environment, separate from the rest of the infrastructure.

  3. 13:43

    The experts read.

    Network, infrastructure, application, logs, hosts and vulnerabilities: each queries only its own tools, read-only.

  4. 13:46

    The orchestrator reasons.

    It lines events up in a timeline, looks for the cause with the 5 Whys and formulates hypotheses.

  5. 13:48

    The report arrives.

    The hypotheses with their confidence level and a proposed solution, for the person on call. Their feedback goes back to the agent.

Illustrative example

Every answer, another why.

The orchestrator does not follow a script: after each answer from the tools it decides what to ask next. It goes down one why at a time, drops the leads that don't hold and stops at the cause.

Store: slow tills since 13:40

  1. Why?Why are the tills slow?

    The store reports slow tills since 13:40. I start from the symptom: how slow is the application, and since when?

    Application

    dynatrace.response_time(service="checkout", window="60m")

    Response times rising since 13:38. Requests are queuing.

    Ruled out · Network

    I rule out the simplest cause first: could it be the network to the store?

    wug.packet_loss(site="store", window="30m")

    No packets lost, latency normal. The network does not explain the slowdown.

  2. Why?Why is the application slowing down?

    If the application slows down and the network is healthy, something is failing behind it. I look for errors around 13:38.

    Logs

    splunk.search(index="store", query="timeout", window="30m")

    14 timeout errors at 13:41, all towards the database.

    Ruled out · Infrastructure

    Timeouts towards the database. Is the server under strain?

    checkmk.host_status(host="db-01")

    Reachable, CPU and memory below thresholds. Not a resource problem.

  3. Why?Why are database requests timing out?

    Healthy server but requests timing out: why? I check open connections, with a command from the allow-list.

    Hosts

    ssh.run(host="app-01", cmd="ss -s")

    The connection pool has been at its limit since the service restart at 13:35.

    Ruled out · Vulnerabilities

    One doubt remains: could a vulnerability explain it? I look up only IDs and versions, nothing else is sent.

    cve.lookup(product="app-runtime", version="installed")

    No critical CVE on the versions in use. Hypothesis unrelated.

Root cause

Most likely cause: the database connection pool is saturated after the release. High confidence. I prepare the report with the timeline and the solution, and wait for feedback from the person on call.

Illustrative example · every action is a read-only request and stays in the audit log

The person on call gets a hypothesis, not an alarm.

The report does not stop at the symptom: it lists possible causes by confidence, shows the timeline and proposes a solution. The on-call person decides what to do and tells the agent whether it was right.

Discover Agents

Incident report

Root cause hypotheses

  • Database connection pool saturated after the releaseHigh confidence
  • Spike in requests on the application serviceMedium
  • Network latency towards the storeLow
  • CVE on an application libraryUnrelated

Proposed solution

Raise the pool limit and restart the service in the agreed window.

On-call feedbackCause confirmed

Illustrative example

It sees everything, it changes nothing.

On an infrastructure you cannot risk touching, the agent is built to read and nothing else.

Read-only
Each expert only uses the tools on an allow-list. None of those tools modifies, restarts or configures anything.
Isolated container
Each investigation runs in its own environment, separate from the rest.
Everything in the audit log
Every request to the tools is recorded.
No internet access
The exception is CVE IDs and versions, sent to authorized domains.
AION model on site
The reasoning comes from an AION model running on the PGX: incident data stays where it is.

Let's bring AION SRE to your monitoring tools.

Tell us which tools you use: we'll show you how the experts read them and what comes out of an investigation.

Office
Corso Vercelli 89, 28100 Novara

Your data is treated according to our Privacy Policy and will not be shared with third parties.