OpenAI just published a case study on Circles, the Singapore-based telco platform company, and the numbers are eye-catching: 22% increase in ARPU, 9% churn reduction, 65% autonomous resolution rate for customer support. These are production results from a regulated, high-stakes industry where AI hype usually dies on contact with reality.
But read the case study closely and you'll notice something frustrating: it's a showcase, not a blueprint. The results are real, the company is real, but the architectural details that would let you evaluate how they got there—or whether you could replicate it—are mostly absent. This is OpenAI's case-study pattern, and it's starting to grate.
What Circles actually built
Circles operates a SaaS platform for digital telcos across 14 countries and runs its own consumer business, Circles.Life, in Singapore. The core innovation is an "AI Concierge" powered by the OpenAI API—a conversational interface that replaces traditional IVR menus and fragmented support workflows.
Behind the concierge sits CareX, a proprietary multi-agent architecture. An orchestration agent routes requests to specialist agents handling billing, subscriptions, network management, and account services. The system is context-aware: it understands browsing history, account state, and current activity before routing a request.
The specialist agents have scoped access—they only see the customer data needed for a specific task, not the full profile. PII is identified and encrypted before reaching the model layer. When the AI can't resolve something, it escalates to a human with full context attached.
For personalization, Circles built Xplore IQ, an OpenAI-powered recommendation engine that combines behavioral signals with real-time context to trigger hyper-personalized actions: roaming pack offers, plan upgrades, churn-prevention nudges.
The numbers are legitimate and notable
Let's be clear: the results are impressive and appear methodologically sound. Circles ran a controlled comparison in Singapore—customers who received AI-powered recommendations versus those who didn't. The AI group saw 22% higher ARPU and 9% lower churn.
CareX hit 55% autonomous resolution in its first week with one operator and now sits at 65% across supported workflows. Circles is targeting 95% autonomous resolution as they expand into real-time voice.
They also used Codex internally for engineering acceleration—29% improvement in development efficiency across design, coding assistance, and unit testing.
These aren't demo metrics. This is a production system handling real customer conversations in a domain where mistakes have regulatory and revenue consequences. That matters.
What the case study doesn't tell you
Here's where my frustration kicks in. If you're an AI engineer or product leader trying to learn from this, the case study gives you almost nothing actionable:
Model and architecture specifics
- Which OpenAI models power
CareX?gpt-4?gpt-4-turbo?gpt-3.5-turbo? - How is the orchestration agent structured? Is it a single model with routing logic, or a classifier feeding into specialist endpoints?
- What does "specialist agent" mean in practice? Fine-tuned models? Prompt-engineered system messages? Separate API endpoints with context injection?
- How do they handle state management across a multi-turn conversation?
Cost and latency
The case study mentions "balancing cost and latency to maximize value" in a quote from Awais Malik, Circles' Global Chief Growth Officer, but provides zero specifics:
- What's the average cost per support interaction?
- What's the p95 latency for autonomous resolution?
- How do they prevent runaway token usage in multi-agent routing?
- What's the trade-off between model capability and cost at 65% autonomous resolution? Would
gpt-4get them to 75% but double the cost?
The PII encryption layer
Circles "identif[ies] and encrypt[s] personally identifiable information before it is passed to LLMs." This is the right approach, but:
- How do they identify PII in real-time conversational context? Regex? A separate model? Deterministic rules?
- What encryption scheme do they use? How do specialist agents decrypt only the scoped data they need?
- Does this PII scrubbing break model performance on edge cases where the actual data matters for resolution?
The 95% autonomous resolution claim
Circles is "targeting 95% autonomous resolution across the entire workflow, including text and voice." That's an extraordinary goal. But:
- What's the current coverage? Are there entire workflow categories excluded from the 65% figure?
- How do they define "resolution"? Customer satisfaction? Issue closed? No follow-up within 7 days?
- What's the false-positive rate—cases marked resolved but actually requiring escalation?
Why this pattern is a problem
OpenAI's case studies consistently follow this template: impressive company, impressive results, minimal technical detail. I get it—there are competitive and partnership reasons to keep some things vague. Circles has proprietary IP in CareX and Xplore IQ. OpenAI doesn't want to hand competitors a cookbook.
But the result is a case study that functions more as a sales signal than a learning resource. It tells potential enterprise customers "yes, this is possible" without giving practitioners enough to evaluate feasibility, cost, or architectural fit.
Compare this to how Anthropic handled the Constitutional AI paper or how Meta released Llama 2 with detailed ablations. You could critique the methodology, run your own experiments, and learn from their decisions even if you disagreed with them.
OpenAI's case studies don't give you that. They're testimonials with numbers attached.
What I'd actually want to know
If OpenAI wanted this to be a genuinely useful case study for the AI engineering community, here's what I'd ask for:
- A technical appendix with model choices, prompt patterns, and agent routing logic—even if redacted or simplified.
- Cost-performance curves showing trade-offs between model tiers and autonomous resolution rates.
- Failure mode analysis—what are the top 3 reasons
CareXescalates to humans? What percentage of escalations are false positives? - Latency budgets—how much time does PII scrubbing add? What's the latency distribution for multi-agent routing?
- Comparative baselines—what did the 65% autonomous resolution replace? A 20% rule-based bot? 40% with traditional NLU?
These details wouldn't compromise Circles' competitive position. They'd make the case study credible and educational.
The broader takeaway
Circles clearly did something right. The results are strong, the multi-agent architecture is sound in principle, and the focus on governance and scoped access is exactly what production AI needs.
But OpenAI's case-study problem is that they're optimizing for marketing credibility—"look, real companies are shipping this"—not engineering credibility. And for a company positioning itself as the infrastructure layer for enterprise AI, that's a missed opportunity.
If you're shipping AI in production, you need to know not just that something worked, but how, why, and at what cost. OpenAI's case studies keep giving us the first part and withholding the second.
I want to learn from Circles' success. Right now, all I can do is admire it from a distance.