In part one, I argued that AI is breaking the assumptions enterprise networks were built on. The network can carry AI traffic, but it can’t tell what that traffic is. In my second blog, I made the case for three layers working as one architecture, not an additional control plane. SD-WAN, SASE, and cloud connectivity remain essential and the missing piece sits above them.
This final post shows what one architecture looks like as a product, what each layer does, and what it changes for the enterprises running AI in production.
The AI-grade network: three layers, one operating model.
Three layers, three jobs
We call it the AI-grade network: three foundational layers, with NetOps for AI operating across all of them as one system.
The network gets the bytes there. Intelligent NaaS makes the path programmable. The AI Fabric understands what the bytes mean: which model, which agent, which data, and at what cost. A programmable network is the basics. An intelligent, operable network is the differentiation.
The design principle that matters the most: the AI Fabric travels with the workload, not the pipe. An AI request shouldn’t be governed differently because it happened to ride a different carrier’s circuit that day.
NetOps for AI: decisions in minutes, grounded in data
When an AI experience slows down, four teams observe four scenarios.
The network team sees a healthy link.
The cloud team views a healthy region.
Security detects no violations.
AI operators perceive a healthy model.
Nobody and no group own the problem. NetOps for AI replaces the disagreement on the issue with a decision, which rests on measurement.
- Observe. A live topology across data center, cloud, and edge, with path health, token-level performance, and cost telemetry in one view.
- Diagnose. NetOps catches sudden anomalies and slow drift against baselines, then correlates network, cloud, security, and model signals into ranked root-cause hypotheses.
- Recommend and act. The system proposes a fix, such as an alternate endpoint or path, and executes it where the customer has enabled automation.
- Validate. Insights can confirm that performance or policy was actually restored.
The speed comes from the data. A software-only gateway can tell you a call was slow but it can’t tell you why. Only a provider that sees the bytes and the tokens in one place can say whether a slowdown is the circuit, the cloud path, or the model.
The control point customers have been missing
Today the questions that matter fall between teams. Which apps are calling AI? What data left in a prompt? Did the model, the network, or a rate limit slow this down? Which team is driving token spend? Each team sees a slice, and nobody owns the answer.
The AI Fabric gives the enterprise a single control point on the path between users, agents, and models. Every request passes through it, so the enterprise can:
- See who is calling which model, from which app, at what cost.
- Govern through policy that is enforced rather than documented.
- Optimize by routing on cost, latency, and sensitivity.
The AI Fabric sits on the path between users, agents, and models, so every request passes through one place that can see, govern, and optimize it. It works across carriers, clouds, and model providers, so the enterprise keeps its freedom of choice. And because AT&T owns the path underneath, the AI Fabric can see inside the network rather than only route around it. Overlay-only players can’t do that, and hyperscalers only see their own cloud.
The value: performance, security and governance, and cost
The AI-grade network pays off on three areas, with each having the same capability: seeing AI activity and acting on it from the path every request already takes.
Performance and reliability
The value: faster answers, fewer outages, and a shorter path from symptom to fix.
- Root cause in one view, not four. Path health and token-level performance (time to first token, tokens per second, errors) sit side by side. A slow AI response is traced to the circuit, the cloud path, or the model, instead of being debated across teams.
- Reroute around degradation. When a provider rate-limits or slows down, the AI Fabric moves the affected call to an alternate endpoint under existing policy, and NetOps validates that performance recovered.
- Repeat requests stop paying full price. Caching serves repeated requests without a round trip to the model, which cuts latency and cost on high-repeat workloads such as FAQ-style assistants.
- A performance floor where it matters. Service guarantees on the inference path give latency-sensitive workloads, such as live video or voice assistants, predictable performance instead of best-effort internet.
Security and governance
The value: enforced policy instead of documented policy, and a record to prove it.
- See the AI you didn’t approve. Discovery inventories the models, agents, and tools in use from live traffic. Unapproved ones can be blocked, and every denied call is logged with a reason.
- Stop sensitive data at the prompt. Inline inspection redacts or blocks account numbers, personal data, and source code before they leave for an external model and records the event.
- Bound what agents can reach. Each agent gets an explicit allowlist of models, tools, and APIs, and everything else is denied by default. A compromised agent can reach only what it was approved for.
- One policy across every path. The same rules apply whether traffic rides AT&T fiber, a partner’s last mile, or another carrier, and across every cloud and model provider.
Cost control
The value: AI spend that can be explained, capped, and steered.
- Attribution. Spend is broken out by workload, team, and business unit, so it can be charged back instead of sitting in one pooled line.
- Budget guardrails. Teams gets a monthly cap and requests over the limit are blocked with a clear reason instead of appearing on next quarter’s bill.
- Model tiering and cost-based routing. Routine requests such as classification and FAQ answers go to a smaller, cheaper model. The frontier model is reserved for the work that needs it.
- Modeled impact. In our internal scenario modeling, placement alone gains about 15%, and model choice plus placement gains about 45%. These are modeled figures, not measured customer results.
What customers can solve for: one incident, two outcomes
Architecture diagrams don’t make the case. Incidents do. Let’s assess the incident laid out in the second blog: the bank with two initiatives that were stress testing their hub-and-spoke network configuration. Now let’s run the same failure twice: on the network it has today and on an AI-grade network.
The incident
Small-business loan decisions are supposed to take minutes. On a Tuesday morning, they start missing the SLA. Applicants are dropping out. The decision workflow fans out to five models and API calls across two clouds, and one of them is the slow one. Nobody knows which.
Models fail; that’s not the point. Models fail. Instead, with an AI-grade network, the team can focus on how long it takes to know which part failed, and who has the authority to act on it.
Same bank, different failure
The bank’s ATM video-banking sessions start to stutter at a branch running on fixed-wireless failover. Token telemetry shows the model responding normally, while path data shows jitter above the session budget. NetOps shifts the session to a healthier path and confirms time to first token has recovered. The bank doesn’t lose a day blaming the AI team. A tool that only sees tokens would have blamed the model. A tool that only sees circuits would have declared the link “up.” In this scenario, the bank avoids:
- Shadow AI. Discovery shows a team piping customer notes into an unapproved model. The bank blocks it and routes the team to an approved alternative, before an auditor finds it.
- A runaway agent. The loan-decisioning agent is cleared for two approved models and one credit-bureau API. When a prompt injection pushes it toward an unfamiliar destination, the default-deny rule stops the call, and the kill switch revokes its access.
Three incidents, one pattern: the problem shows up in one team’s dashboard and gets fixed in another dashboard. The AI-grade network closes that gap.
What gets built on top
A control point on the path is a platform and these are the directions we are building on it:
- Agentic security. An authenticated agent is not necessarily trustworthy. A prompt-injected agent can look healthy while compromised. Controls like agent identity and a registry, per-action authorization, trust decisions on tools and MCP servers, and a machine-speed kill switch have to sit on the agent’s path to models and tools. Agents also need proof that models and data ran untampered. TEE attestation ensures proof and signed receipts so regulators can verify independently.
- Inference routing with explicit constraints. Constraints can look like: “This data class never leaves the region,” “Stay inside a 200 ms time-to-first-token budget,” or “Use the smallest model that clears the accuracy bar within budget.”
- Closed-loop autonomy. As decisions prove out, customers can move NetOps from recommend to act for more classes of incidents.
Where this leaves us
The next decade of enterprise networking will be won by understanding the traffic: what it is, what it costs, and whether it should be allowed. The AI-grade network pairs the physical network and Intelligent NaaS with the AI Fabric, and NetOps for AI operates across all three layers.
AT&T Business has opened an MVP program to help us build a solution to:
- See who is calling which model, from which app, at what cost.
- Govern with policy that is enforced rather than documented.
- Optimize by routing on cost, latency, and sensitivity.
Read more AT&T Business news
