Build your own AI agents: the advanced path

Before you start

This path assumes you’re comfortable with assistants, consumer AI agents and basics such as tokens, context and benchmarks. If not, the Go further path will get you there. Here, each step matches a concrete decision in the life of an agent project, from idea to production. The tools mentioned run entirely in your browser, using dated public data refreshed every week.

The 12 steps

Step 1 — Define your agent’s task precisely

A successful agent starts with a well-defined task: what it receives, what it should produce, which tools it can use and, above all, what it must never do. Write down your success criteria too: how will you recognize a good result? These elements become the foundation of your instructions (the “system prompt”) and your tests.

→ To do: draft a first version of your agent’s instructions with the prompt generator.

Step 2 — Choose between no-code and code

No-code automation platforms connect apps and AI steps with drag and drop: ideal for fast prototyping and simple workflows. Code, using a framework (a library that provides the building blocks of an agent), gives you more control: custom logic, automated tests, fine-grained handling of errors and costs. Many projects start with no-code and migrate once the logic gets more complex.

→ To do: list your project’s constraints (privacy, volume, available skills) before deciding.

Step 3 — Pick a framework

Agent frameworks differ in language, philosophy (single agent or multi-agent), license and how active their community is. Look at the date of the latest release, the quality of the documentation and how easily you can swap the underlying model: a framework that locks you into one provider can get expensive later.

→ To do: compare your options in the agent frameworks explorer, which shows GitHub stars, license and latest release.

Step 4 — Choose your agent’s model

The model is the agent’s engine. The criteria differ from chat use: reliable tool calling (the ability to correctly trigger an external function), sticking to output formats, context window size, speed and price. A smaller, cheaper model is often enough for simple subtasks.

→ To do: put your candidates side by side with the compare models tool, then see where they stand in the AI ranking.

Step 5 — Estimate the cost

Models accessed through an API (an interface that lets a program call the model) are billed per token, with different prices for input (what you send) and output (what the model generates). An agent that chains steps often resends the whole history on every call, so costs climb faster than you’d expect. Estimate before you launch, then compare with reality.

→ To do: measure the size of your instructions with the token estimator, then simulate your budget with the agent cost calculator.

Step 6 — Manage context

The context window is the maximum amount of text a model can take into account at once. When your documents exceed it, you use RAG (retrieval-augmented generation): the agent first searches your knowledge base for the relevant passages, then sends only those to the model. Summarizing the history and filtering what goes into the context improves both quality and cost.

→ To do: check whether your documents fit in your target model’s context with the context window tool.

Step 7 — Connect your tools with MCP

MCP (Model Context Protocol) is an open standard for exposing tools and data to an agent in a consistent way: the same connector can work with several compatible apps. Before adding an MCP server, check who published it, which actions it allows and which data it can reach. Prefer read-only connectors unless write access is truly required.

→ To do: list the access your agent actually needs and mark each one as read-only or write.

Step 8 — Run a model locally

Running an open-source model on your own machine or server keeps your data in-house and removes pay-per-use billing. In return, you need suitable hardware, especially GPU memory, and must accept performance that’s sometimes lower. Quantization (reducing the precision of the numbers that make up the model) lets you run larger models on modest hardware, at the cost of a slight drop in quality.

→ To do: find models that suit your hardware and license needs on the open-source models page.

Step 9 — Secure your agent

An agent that reads external content is exposed to prompt injection: hidden instructions that hijack its behavior. Apply the principle of least privilege (the agent only gets the permissions it strictly needs), clearly separate instructions from external content, add guardrails (output checks, allow-lists of actions) and require human approval for anything irreversible: payments, sending, deleting.

→ To do: run each of your agent’s actions through the “Can I trust an agent with this?” tool to spot the ones that need human sign-off.

Step 10 — Evaluate your agent

Public benchmarks give you a first sense of a model’s level, but they don’t measure your task. Build your own test set: real cases, edge cases and hijacking attempts, each with the expected answer. Rerun it every time you change the model, the instructions or a tool, and track results over time.

→ To do: read the site’s methodology to understand how public scores are collected and what they really measure.

Step 11 — Keep up with research

Agent techniques evolve very quickly: new methods for planning, memory, evaluation and security. You don’t need to read everything: spot the work that relates to your use case and test it on your own test set before adopting it.

→ To do: check the weekly radar each week; it gathers the latest research papers from arXiv and technical discussions from Hacker News.

Step 12 — Go to production

Putting an agent into production means accepting that it runs without you. Plan for detailed logs of every run, cost monitoring with a spending cap, error handling (retries, a fallback model), versioned instructions and a simple way to shut everything down. Start with a narrow scope, observe, then expand.

→ To do: recalculate your budget with real volumes in the agent cost calculator, and follow AI news to anticipate price and model changes.

What comes next

An agent is never truly finished: models change, prices move, new risks emerge. The site’s data is refreshed every week, with sources and dates, so you can revisit your choices regularly. Keep the tools page close at hand: it’s your dashboard for revisiting each decision with up-to-date numbers.