In June 2025, Gartner predicted that over 40% of agentic AI projects would be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. In the same release, Gartner estimated that of the thousands of vendors marketing themselves as “agentic AI,” only about 130 are real. The rest are rebranded chatbots and existing automation tools; the industry has a name for it: agent washing.
The implication for mid-market leaders is uncomfortable. A wave of investment is being directed at the most expensive, most complex tier of AI architecture, often by buyers who don’t yet have a clear picture of what an agent actually is, what it costs to operate, or whether their use case requires one. Most don’t.
The previous post in this series argued that the right unit of analysis isn’t the workflow; it’s the component. Once you’ve decomposed correctly and chosen a component worth automating, there’s a second over-engineering trap waiting: choosing the wrong complexity tier to build it at. This post is about how to pick the right one.
The Four Tiers
Most AI deployments fall into one of four architectural tiers, ordered from simplest to most complex.
Rules and spreadsheet logic. If/then statements, lookup tables, formulas, deterministic scripts. Fast, cheap, predictable, and auditable. The tradeoff: rules don’t generalize. They handle the cases you anticipated and break (or escalate) on the ones you didn’t. For tasks where the variation is bounded and the logic is stable, this tier is often all you need; no LLM required.
A single LLM call. One prompt, one response. The model classifies, extracts, drafts, or summarizes; a human or downstream system uses the output. Flexible enough to handle variety, cheap enough to scale, simple enough to debug. The Anthropic engineering team puts it directly in their guidance for builders: “For many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough.” Most of the AI value being created in mid-market companies right now sits in this tier.
An AI workflow. A predefined sequence of steps, with LLM calls at specific nodes. The path is fixed; the model handles the parts that benefit from language understanding (classification, extraction, generation), and deterministic logic handles the rest (routing, validation, integration). More moving parts than a single call, but still debuggable, testable, and predictable. Most production AI systems doing serious work, billing automation, document processing, structured analysis, are workflows.
An AI agent. The model decides, at runtime, what to do next: which tool to call, what information to gather, when to take an action, when to stop. That autonomy is genuinely useful when the path can’t be predefined; agents can reason through novel situations, synthesize across sources, and recover from failures in ways a fixed workflow can’t. The cost of that flexibility is real: token bills run an order of magnitude higher than a single LLM call, latency is measured in tens of seconds, and failure modes multiply because the system has more degrees of freedom. Reach for an agent when you genuinely need that autonomy. Don’t reach for it because the demo at the conference looked impressive.
The Principle
Only graduate to a higher tier when you’ve proven the lower one doesn’t work.
This is Occam’s Razor applied to AI architecture. Start at the simplest tier that could plausibly handle the task. Ship it. If it fails, identify the specific reason, then graduate up exactly one tier. Don’t skip rungs.
The reason this matters more than it sounds is that the costs of choosing too high a tier are mostly invisible at the planning stage. They show up later: longer time to ship, higher token bills, harder debugging, more places for the system to fail, more change-management friction with the team that has to operate it. Each tier you skip up the ladder, you’re paying these costs whether or not you needed the capability.
What Right-Sized Looks Like: BD Pulse
When we built our internal business development tracker at Areté (BD Pulse), we faced a choice every mid-market company faces when picking a CRM: buy Salesforce or Pipedrive, or build something purpose-fit. We built. The reason wasn’t ideological; it was that an off-the-shelf CRM gives you no control over what data flows through which AI system, and our BD activity touches sensitive client information that we needed to govern carefully.
The interesting decision wasn’t build versus buy. It was where to put the LLMs once we’d decided to build.
Most of BD Pulse is not AI. The contact records, institution profiles, activity logs, pipeline stages, leaderboards, scoring rules, badge logic, calendar views: all standard application code. None of it benefits from a language model.
The AI shows up in three specific places, and each is a single LLM call: when an inbound email arrives at our BD address, the model classifies the activity type, extracts contacts and companies, and generates a one-line summary; when a user asks for research on a prospect, the model synthesizes a brief from the available context; when a pitch deck is uploaded, the model extracts the pitched team members. Each of these is exactly the kind of task where language understanding earns its place. None of them needs an agent.
Built this way, the system shipped in four weeks with one engineer at 0.75 FTE and one AI enablement lead at 0.5 FTE. The cost math worked because our engineering team is AI-native (we use coding agents in our own workflows, which is its own form of right-sizing), but the architecture is what made the build economical in the first place. Once the system was live, it cost almost nothing to operate. And because the LLM only ever sees the specific text we route to it (the email body is processed for metadata, then discarded), we get a level of data control that no off-the-shelf CRM with bolted-on AI features could give us.
If we’d reached for an agent (“an autonomous BD assistant that manages your pipeline”), we’d have spent months instead of weeks, paid orders of magnitude more in token costs, introduced unpredictable failure modes, and gained nothing the simpler architecture didn’t already provide. The agent version would have been more impressive in a demo. It would also have been worse.
The Common Failure Patterns
When we audit stalled AI initiatives, the over-engineering pattern is usually one of three:
The team built an agent when a workflow would have worked. The path was actually predictable; the team just didn’t take the time to map it. They paid for runtime decision-making they didn’t need.
The team built a workflow when a single LLM call would have worked. They added orchestration, retries, and step logic to a problem the model could have solved in one shot with a well-structured prompt and a few examples. The workflow looked rigorous; in practice, it just slowed everything down.
The team built a single LLM call when a rule would have worked. The variation in the inputs was bounded, the logic was stable, and the reason an LLM was used at all was that the team was excited about AI. They’re paying per-call costs to do something a 30-line script would have done for free.
Each of these has a cheaper, faster fix than the rebuild your vendor is going to recommend.
What to Do This Week
For your next planned AI initiative, before any architectural decision is made, draft it at the simplest tier first. Write out what the rules-only version would look like. Then the single-LLM-call version. Only graduate up the ladder when you can name the specific failure of the simpler tier that forces it.
If the simpler version doesn’t fail in any specific way you can articulate, that’s your answer. Build it.
The Bottom Line
The companies that ship AI faster and cheaper than their peers aren’t the ones using the most advanced architecture. They’re the ones who keep asking, at every step, whether something simpler would have worked. The most expensive AI mistake isn’t building the wrong thing; it’s building a sophisticated thing when a simple thing would have worked.
The Anthropic engineering team, the people building the most capable models in the industry, put it this way in their published guidance: “we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.” That last sentence is the one mid-market leaders should print and put on the wall.
This is the fifth and final post in our series on the AI Value Map. Across these five posts, we’ve laid out a method built for mid-market companies: discover where AI is already working in your business, score and decompose what you find, get the Big Three right, and resist the pull toward unnecessary complexity. The framework is now in your hands. What will you do with it?
