The demo is a loop. Production is everything around it
You sign up to own the entire system, from the model loop to access control, failure handling, storage, monitoring, and evaluation. Building an AI agent from scratch can teach you how the pieces work, but production means maintaining every surrounding service and policy as the agent takes on real work.
You can learn how to build ai agents from scratch in an afternoon. A while loop, a model call, maybe one tool that fetches live data, and you have the shape everyone talks about. Read input, decide, act, observe, repeat. The model proposes; your runtime executes. That part is teachable and satisfying.
What the tutorials rarely linger on is what happens when the agent leaves your laptop. Someone asks for a refund summary tied to billing data. A scheduled job runs at 6 a.m. and posts to a channel. A tool write fails halfway through a four-step workflow. Suddenly you are not polishing a REPL. You are building software that other people trust with real work.
This post inventories that second layer so you can decide whether from scratch is the right bet for your team right now. I am not trying to scare you off custom code or sell a fairy tale.
The skeleton everyone builds first
Strip the demos down and most agents share the same bones. Instructions tell the model how to behave. Tools extend what it can touch: read a database, send a message, look up a doc, call an API. Orchestration is the loop that keeps calling the model until something counts as done: a final answer, structured output, a handoff to a human, or a stop you defined in code.
Single-agent setups stay sane longer than people expect if tools are distinct and instructions are explicit. Multi-agent patterns help when prompts sprawl with branches or when the model keeps picking the wrong capability among similar ones. None of that is magic. It is still a loop with exit conditions and guardrails you define.
That skeleton is the fun part. The bill arrives when the skeleton has to run where your company already lives.
Auth and access: who is the agent, anyway?
Your agent is not a user with a password. It is a service identity that needs to call Stripe, GitHub, Notion, Gmail, or whatever your workflow touches. Each integration wants OAuth flows, token refresh, scoped permissions, and often separate credentials per environment.
You also need to answer a harder question: what may this agent read, and what may it change? Read-only research is a different risk profile than posting to Slack or updating a record. Production systems usually split those paths. Analysis and retrieval on one side. Proposed writes that wait for a human on the other.
If you build from scratch, auth becomes a small product inside your product. Connectors multiply. Tokens expire on holidays. Someone revokes an app install and your nightly report silently stops until someone notices. Platforms that already wire common tools exist partly because this work is repetitive and easy to get subtly wrong.
Retries, failures, and the unglamorous runtime
Models stall. APIs return 429s. Tool arguments arrive malformed. A step succeeds in the tool but the model misreads the result and goes down a rabbit hole.
You need policies for how many retries to allow, what backoff to use, and which errors qualify. Idempotency for actions that must not run twice. Timeouts so one hung call does not hold a worker open forever. Failure modes that return control to a person instead of looping until someone kills the process.
Deterministic automation hides failure behind "job failed" emails. Agents fail in messier ways because the model can choose a different path on the next turn. Your runtime has to log enough context to reconstruct what happened without replaying the entire conversation from memory.
This is where many homegrown agents feel fine in demo week and brittle in month two. The loop did not change. The error handling did not keep up with reality.
Storage and memory: more than a message array
The tutorial transcript is a list of messages in RAM. Production asks what persists across sessions, across users, and across restarts.
Run history matters for debugging and for compliance. Structured knowledge matters when the agent should answer from your policies, not from generic training data. Embeddings and retrieval add pipeline work: chunking, updates when docs change, permissions so one customer never sees another's data.
Even simple workflows accumulate state. Which step finished? What draft is waiting for approval? What did we already send to the customer? You end up designing tables or documents for things the demo never named.
Connected knowledge layers also need a clear contract with source systems. Read-only analysis against live tables is one pattern. Queued writes that a human approves is another. Mixing those up in one agent without boundaries is how you get confident wrong actions.
Observability: seeing inside the loop
When the output looks wrong, "the AI did something weird" is not a ticket your future self can fix.
You want traces: user or trigger, model turns, tool calls with inputs and outputs, latency, token usage, final outcome. Dashboards help. So do alerts when failure rates jump or when a tool starts timing out.
Agents are harder to observe than plain HTTP services because one "request" might be six model calls and four tool hops. Without instrumentation planned early, you grep logs and guess.
Teams that operate agents like they operate other backend services fare better. Teams that treat the prototype script as production learn the difference quickly.
Evaluation: the part that never feels done
Shipping the loop is day one. Knowing whether the agent is good enough is week ten and forever after.
You need examples of real tasks, expected behaviors, and ways to score outcomes: did it use the right tool, respect policy, produce the right format, abstain when data was missing? Regression sets matter because the next model version or prompt tweak will change behavior in corners you did not touch.
Guardrails add another eval surface: off-topic queries, injection attempts, PII in outputs, risky tool use. Each layer you add needs tests, not hope.
From scratch means you own the harness. Nobody else will tell you calmly that accuracy dropped on refund edge cases after last Tuesday's deploy. That discipline separates a toy agent from something an ops lead will let run on a schedule.
When building still makes sense
Custom code earns its keep when the workflow is narrow, the integrations are yours alone, your team already runs reliable services, and nobody else needs the same glue. Learning the loop deeply makes you a better buyer and a better operator. You will recognize when a framework is hiding complexity versus when it is saving real time.
Building less makes sense when the value is in the workflow and the connections, not in maintaining OAuth for half a dozen SaaS tools. When non-engineers need to adjust steps without opening a pull request. When you want scheduled and triggered multi-step flows without inventing a job runner from scratch.
The honest split: implement the loop once for education if you want. Before you promise production, price auth, retries, storage, observability, and evaluation as line items with owners and timelines.
How AI Agent helps
AI Agent is a no-code platform to build, deploy, and run AI agents that automate busywork: research, workflows, reports, and more. Workflows handle multi-step jobs on a schedule or trigger. Autopilots run on their own. Company Brain holds connected structured knowledge agents read from, with read-only analysis against source tables and proposed writes waiting for human approval. It connects to tools teams already use, including Stripe, PostHog, GitHub, Notion, Linear, Slack, and Gmail.
If your question is not how the loop works but how to get more done without doing more, buying the surrounding plumbing is often the rational move.
The loop is a weekend. The rest is the job.
What each part does
| Component | What it does | What breaks if it is missing |
|---|---|---|
| Auth and access | Controls which services and data the agent can use | Credentials expire and permissions become too broad or unusable |
| Retries and failure handling | Recovers from temporary errors and stops unsafe repeated actions | Tool failures halt workflows or cause duplicate actions |
| Storage and memory | Preserves run history, workflow state, and permitted knowledge | The agent loses context and cannot resume work reliably |
| Observability | Records model turns, tool calls, timing, and outcomes | Teams cannot trace incorrect results or detect silent failures |
| Evaluation | Tests task quality, policy compliance, and behavior changes | Regressions reach users without a reliable warning |
Frequently asked questions
How much does it cost to build an AI agent from scratch?
The article does not assign a fixed price to custom work. The cost comes from engineering and ongoing ownership of authentication, retries, storage, observability, and evaluation. AI Agent pricing starts at $49 for the Start tier, and Pro is $149.
How much ongoing effort does a custom agent require?
A prototype can be built quickly, while production requires owners for credentials, failure handling, state, monitoring, and tests. The team also has to update integrations, review regressions, and decide which actions require human approval. That work continues after the initial loop is finished.
What risks come with giving an agent access to company tools?
The main risks are excessive permissions, incorrect writes, exposed data, and failures that go unnoticed. Read-only analysis, scoped credentials, queued writes, and human approval can limit the impact of a mistake. Logs and traces help the team understand what happened and correct the system.
What tends to break after the demo works?
APIs can time out, return rate limits, reject malformed tool arguments, or complete an action that the model misreads. State can also become unclear when a workflow stops partway through or restarts. Without retries, idempotency, timeouts, and useful traces, the team may see only a generic failure instead of the cause.
What does a platform replace when a team buys instead of builds?
A platform can replace much of the recurring plumbing around the model loop, including connectors, scheduled workflows, runtime handling, knowledge access, and approval paths. AI Agent exposes 40 connections and supports workflows, autopilots, and connected company knowledge. The team still needs to define the workflow, permissions, policies, and evaluation criteria.