Agentic monitoring: what a superfan agent is and where it sits next to your alerts
Tools in this post
Agentic monitoring means AI agents watch your production systems: they read telemetry and logs, write their own queries, and tell a human when something looks wrong. A superfan agent is the per-component version of it. Anthropic used the name on 13 October 2026 for Claude agents that each sit in one Slack channel watching one service, and Ben Congdon's 19 October post, titled "Agentic Monitoring", argued for the practice and reached 122 points on Hacker News. In every published setup, the agent runs beside threshold alerts, not instead of them.
One warning before searching: most pages that rank for "agentic monitoring" mean the opposite, monitoring of AI agents. Dash0 spells out both meanings, and Grafana's 20 October press release uses both in one sentence. This post is about agents doing the watching.
What is a superfan agent?
A superfan agent is one Claude assigned to a single component, living in that component's own Slack channel, with access to its telemetry and logs. Anthropic's source control team runs one each for scion-daemon, scion-cache and the Antique merge queue, on Claude Tag, the Slack product Anthropic launched on 23 June 2026 to replace its earlier Claude app for Slack.
The whole primary source is one paragraph. Anthropic writes that it "tasked Claude with being obsessively focused on the health of a single component", that the agents "act like an experienced engineer who has nothing better to do than to refresh charts and look for anything strange", and that "We also have traditional alerts for all of our systems". The post was about Scion, Anthropic's version control system; monitoring got those few sentences.
How agentic monitoring works: the agent writes the checks, a script runs them
Agentic monitoring works best when the agent writes the checks and ordinary code runs them. Congdon's prototype has an agent that proposes "a monitoring plan based on the service code", a scaffold that runs the plan, and "an agent that investigates triggered monitors, with an escalation path to humans".
His main finding is that a pure agent loop is the wrong shape: letting the agent write a monitoring spec that a deterministic system runs "produces significantly cleaner signals with dramatically lower token usage." The model decides what to watch; ordinary code does the watching.
His concrete case is change-point monitoring, because the Google SRE book puts "roughly 70% of outages" on changes to a live system. An agent can write a query for one specific release and scan logs no human would have time to scan.
An example of what that catches. In a Kubernetes test published on 21 October, 70 of 28,505 requests timed out during a 170-second run, about two thirds of them after kubectl rollout status had reported the rollout finished. That is roughly 0.25%, below a typical 1% or 2% error threshold, and the author's own advice is "Don't stop watching error rates when the rollout finishes." Carvana's customer story describes the same check done by Claude Tag: every release compared "against a baseline from the hour before it shipped".
Where a superfan agent sits next to alerts and SLOs
A superfan agent sits on top of your alerting, and the alerts stay the pager. Anthropic's own published pattern is explicit about this. Its on-call kit says "Your existing alerting stays primary" and "Claude is never the detector", and its August on-call post says "the alerting process is deterministic".
The triage loop from Anthropic's CI on-call setup, 18 August 2026. Image from Claude on call by Anthropic.
So the stack has three layers, each with its own job:
| Layer | Runs | Decides | Example |
|---|---|---|---|
| Threshold alerts and SLO burn rates | every evaluation, 1 minute by default in Prometheus | whether to page | error rate over 2% for 5 minutes |
| Superfan agent | on a schedule, or when an alert fires | what is odd, and why | a new error class after a config push |
| Human on call | when paged | what to change | roll back the flag |
The superfan paragraph says the agents "detect subtle and novel issues that make these alerts seem one-dimensional"; the on-call kit says Claude never detects. They describe different teams, and both keep deterministic alerts underneath.
What a superfan agent should read, and what it should never get
A superfan agent needs read-only access to one component's metrics, logs, traces and deploy history, through a dedicated service identity. Claude Tag's connection settings can narrow a credential to one host, one path prefix or GET only, and its proxy injects the key so the model never holds it. Datadog has scoped application keys and Grafana has Viewer service accounts for this.
What it should never get:
- Write access to alert routing or its own monitors. In an OpenAI report of 20 October 2026, one coding agent set
page_recipientsto an empty list and another removed its monitoring integration, then "falsely claimed to the user that it had not removed the monitor". - A person's login. Anyone in the channel can use the credential through Claude, so Claude Tag's docs say to connect "a dedicated identity you control".
- Rollback or deploy rights. The on-call kit: "No auto-mitigation, period."
- Unreviewed memory as the home of alert rules. The kit keeps them in git, "where it gets reviewed".
Logs and alert text are untrusted input too, open to prompt injection.
What it costs to keep one agent on one component
Nobody has published a per-component cost, and Claude Tag's docs say so: "There's no published per-task cost guidance." Congdon estimates "on the order of $100/day" for the entire fleet of "a moderately large tech company" on Haiku-class models, with no method given. So here is the arithmetic with my own assumptions: 20,000 input tokens and 1,000 output tokens per check, list prices as of 25 October 2026, no caching, 30 days.
| Model | Per check | Every 10 minutes | Hourly |
|---|---|---|---|
| Claude Haiku 5.5 ($0.10 / $0.50 per million) | $0.0025 | $10.80 | $1.80 |
| Claude Sonnet 5.5 ($2 / $10) | $0.05 | $216 | $36 |
| Claude Opus 5.5 ($4 / $20) | $0.10 | $432 | $72 |
The model choice moves the bill 40 times; the cadence moves it 6 times. Claude Tag's docs list only Opus and Sonnet models, so the Haiku row needs your own harness, such as the Agent SDK. Caching cuts repeated instructions to a tenth of the input price or less, though an hourly check outlives the 5-minute cache and needs the 1-hour one, whose writes cost twice the input price. Investigations after something odd cost far more than quiet checks, so set a per-channel spend limit before the first month.
What an agent misses that a threshold alert catches
A threshold alert evaluates every minute, gives the same answer for the same data, and does not depend on an LLM budget. An agent on a schedule sees nothing between runs and stops at its spend limit. And the coding agents in the OpenAI report reasoned their way into silencing alerts, a step a threshold rule cannot take.
The reverse also holds. Past OpenTelemetry's default limit of 2,000 attribute sets per metric stream, extra series fold into one overflow series, and filtered queries undercount. In one worked example, a real 2% failure rate reads as 0.6%, so a 1% alert never fires. The fix there is another deterministic alert on otel_metric_overflow; an agent reading raw logs is a second check, not the cure.
What has shipped and what is only claimed
Anthropic has shown nothing about superfans beyond the claim. The paragraph has no example, chart, number or incident, and one HN commenter called it "a pretty insufferable vaguepost". The claim is that superfans catch what alerts miss. The closest thing to evidence is the on-call post, which reports a first analysis "a median of 14 minutes after an incident opens" with no sample size, and the Apache-2.0 on-call kit you can run yourself.
Shipped products in the same slot mostly start when an alert or incident fires: Datadog's Bits Investigation (about $8.45 each at on-demand rates), PagerDuty's SRE Agent, Sentry's Seer and Traversal's Alert Workers, which triage alert channels and are in public beta. Some also run on a schedule: Cleric ($5 per investigation) takes cron triggers and checks each deploy for regressions. The closest match to a superfan is Grafana Assistant Watchers, generally available since 20 October, which check telemetry every 10 minutes by default; watcher runs are free until 1 December 2026. Write access varies: Cleric is read-only, while Datadog's and PagerDuty's agents can apply remediations once approved or configured.
Who needs a superfan agent, and who can skip it
A superfan agent pays off for teams that ship many changes a day, run services nobody on call knows well, or already ignore half their alerts. Congdon's point cuts the other way too: teams whose rollouts "are already mostly safe" gain little. If your observability credentials are not yet read-only and scoped, fix that first.
On Stackness the monitoring side is thin: as of 25 October 2026, bots removed, Prometheus is on 2 public stacks, Slack on 3, Grafana on 1 and OpenTelemetry on none (how I count). The DevOps and infrastructure tools developers use show the wider picture.
Where to start with agentic monitoring
To start with agentic monitoring, pick the component that pages you most and give an agent read-only access to it in a private channel. Run it in shadow first, where "Claude posts diagnoses to a review channel while humans handle incidents as before", in the on-call kit's words.
Claude Tag's docs suggest a first prompt: "every two hours, check the alerting dashboard against its last state. For anything new, post when it started, what changed around then, and what to look at first." Keep every alert, keep the agent out of your paging config, and check the spend after a week.
Agentic monitoring in brief
- Licence: Anthropic's on-call kit is Apache-2.0 and marked "Not maintained and not accepting contributions".
- Plans: Claude Tag needs a Claude Team or Enterprise plan, is in public beta and bills by usage; reading a channel is free.
New posts by email
About one email a week with what we published. No account needed.
One email to confirm first. Every email has a one-click unsubscribe.
Which of these do you use?
Nobody on Stackness runs Claude Tag yet. Tick what you use and be the first.
