🤖 How to build AI Agents from scratch in 10 steps

Welcome back! Ten steps from idea to shipped agent, and three deep dives into what's running underneath all of it. 

In today's Generative AI Academy Newsletter:

That's exactly how you build AI agents in 2026.

Ten steps, start to finish:

1️⃣ Define the role and goal: what it does, who it helps, what it outputs.

2️⃣ Design structured input and output: schemas over messy free text, thinking like an API.

3️⃣ Tune behavior and add protocol: role-based prompts, standardized handling via MCP.

4️⃣ Add reasoning and tool use: frameworks like ReAct or Chain-of-Thought, plus access to search and document tools.

5️⃣ Structure multi-agent logic if needed: separate roles like Planner, Researcher, Reporter, each with its own schema.

6️⃣ Add memory and long-term context: conversational, summary, or vector-based memory so nothing gets forgotten between sessions.

7️⃣ Add voice or vision, optionally: letting the agent see and speak.

8️⃣ Deliver the output: formatted, readable, parsable.

9️⃣ Wrap it in a UI: this is the step that turns an agent into an actual product.

🔟 Evaluate and monitor: test prompts, logs, benchmarks, feedback loops.

Before any of that makes sense, it helps to understand what these tools do in plain terms first. 

"What Can AI Actually Do For You?" is a free 30-minute course covering five practical use cases and how to pick your first tool.

Seven Claude courses at GenAI Academy. All recorded, and all free.

Start with "What Can AI Actually Do For You?" if you're new, 30 minutes, no setup required. 

Then the Claude Starter Course gets you running in 35.

Past that, pick by the problem you have. Hitting usage limits in Claude.ai, Cowork or Claude Code has its own course. 

Building no-code agents has one. 

So does rolling Claude out across an ops team in 30 days, from pilot to adoption data to full deployment.

The AI Portfolio Builder turns whatever you built into a case study you can show someone.

World Wide Vibes Hackathon is live too, $5,000 prize pool, beginner-ready, 100% online.

One login. Watch them in any order.

Browse the free Claude courses.

9 neural network concepts, with the math worked out

1. What an ANN is
Neurons arranged in layers. Input, one or more hidden, output. That's the whole structure.

2. The artificial neuron
Takes inputs, multiplies each by a weight, adds a bias, runs the result through an activation function. z = w1x1 + w2x2 + w3x3 + b, then a = f(z). Everything else is this repeated.

3. Forward propagation
Worked example with one neuron. Inputs 2 and 3, weights 0.5 and 0.2, bias 0.1.
z = (2 × 0.5) + (3 × 0.2) + 0.1 = 1.7
ReLU(1.7) = 1.7
That value goes to the next layer.

4. Activation functions
ReLU: max(0, x), the default in hidden layers.
Sigmoid: squashes to between 0 and 1, useful for binary classification.
Tanh: between -1 and 1.
Without one of these, stacking layers gains you nothing, because linear on top of linear is still linear.

5. Loss function
Measures how wrong the prediction was. Binary cross-entropy with a true label of 1 and a prediction of 0.8 gives L = -log(0.8) = 0.223. Lower is better.

6. Backpropagation
Gradients of the loss with respect to every parameter, computed backward through the network using the chain rule. Then the optimizer moves each weight against its gradient.
w_new = w_old - η(∂L/∂w)

7. The training loop
Dataset, forward pass, loss, backpropagation, weight update. Repeat for many epochs.
One update: w = 0.50, gradient = 0.20, learning rate = 0.10, so w_new = 0.48.

8. Architecture choices
Fully connected layers, with the output layer determined by the task. Regression takes linear. Binary classification takes sigmoid. Multi-class takes softmax, which turns logits [2, 1, 0] into roughly [0.665, 0.245, 0.090].

9. Where they get used
Tabular prediction, spam classification, house price regression, churn prediction, fraud detection, demand forecasting, risk scoring.

Learn the forward pass, the loss, backpropagation and optimization in that order. 

Everything else in deep learning is built on those four.

Spent the past few months on this. Here's the path that actually worked.

1. 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝐬
Goal: know what happens when you make an LLM call.

Tokenization, embedding, forward pass. Autoregressive generation. Prefill and decode. KV cache. TTFT and ITL. Throughput against latency.

Start with NVIDIA's talk at AI Engineering, then Hugging Face's inference guide.

2. 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐭𝐡𝐚𝐭 𝐦𝐚𝐭𝐭𝐞𝐫 𝐡𝐞𝐫𝐞
Goal: understand the computation, and only the parts inference touches.

Transformer block, embeddings, self-attention, Q, K and V. The bbycroft LLM visualizer is worth an hour on its own.

3. 𝐆𝐏𝐔 𝐟𝐮𝐧𝐝𝐚𝐦𝐞𝐧𝐭𝐚𝐥𝐬
Goal: understand the constraints underneath every performance number.

GPU architecture, streaming multiprocessors, HBM against SRAM, memory hierarchy, bandwidth, FLOPS, and the difference between compute-bound and memory-bound work.

Horace He's "Make Deep Learning Go Brrr" is required reading. Overhead, memory bandwidth and compute are the three bottlenecks, and it teaches you which one you're actually hitting.

4. 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐭𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬
Goal: understand how modern systems get fast.

Quantization, PagedAttention, KV cache quantization, FlashAttention, chunked prefill, speculative decoding, prompt caching, continuous batching.

NVIDIA's "Mastering LLM Inference" covers the landscape. Then read the original paper for whichever one you need to go deep on.

5. 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐞𝐧𝐠𝐢𝐧𝐞𝐬
Goal: see how they differ, because each wins on a different workload.

vLLM is the best general purpose option. Battle-tested and built for high-throughput serving.

Start at section 3 if you already build with these. Understanding the memory bottleneck changes how you read everything in section 4.

Your CV gets ten seconds.

That's how long a hiring manager spends matching you against a job description. They scan for their language, not yours. Miss it and you're skipped, however qualified you are.

So the rejection usually wasn't about your experience.

The fix is rewriting the CV per application, which nobody does across ten roles. ChatGPT closes that gap. Paste in your CV and the job description, and twenty minutes gets you a tailored version, a LinkedIn summary and a clear read on where you're strong.

Four ways to push it further

Run it on a role you already got rejected from, using the CV you actually sent. The difference usually explains the outcome.

If three job descriptions name the same missing skill, that's a pattern worth acting on.

Changing industries? Use it to translate your experience into the new field's vocabulary.

Feed the tailored CV and job description back in for a cover letter draft. It won't be finished, and it beats a blank page.

One caution. Reframing in their language is the point. Inventing experience is a different thing, and it surfaces in the interview.

A CNN assumes nearby pixels relate. An RNN assumes order matters. A Transformer assumes anything can relate to anything.

Match the assumption to your problem and most of the work is already done.

The short version of picking one

Images → CNN
Sequences → RNN, LSTM or GRU
Large-scale language → Transformer
Tabular → start with a plain ANN

The deeper point is that you stop hand-crafting features. The model finds them, which is the whole reason this replaced classical machine learning for messy data.

And almost every modern AI system is a variant of something on that list rather than a new idea.

Five things that save you time

→ Start simpler than you think you need, then add complexity when the simple version fails for a reason you can name
→ Get enough data, and use dropout, because your model will happily memorize the training set
→ Compare two architectures on the same data before committing. Cheaper than being wrong for three months
→ Use a pre-trained model unless you have an argument against it
→ Keep reading, since the defaults change about every eighteen months

Full formulas and diagrams in the image.

Follow us on Instagram for fast, visual AI updates in 30 seconds. 

Subscribe to our Atlas newsletter — trusted by 3M+ subscribers — to stay ahead of AI news across tech, education, and business. 

📺 Watch us on YouTube to hear insights directly from leading AI voices, builders, and innovators.

🐦 Follow us on X for breaking AI news and real-time industry updates.

Learn how to build your next AI application with practical resources and expert guidance.

Explore investment opportunities in the future of AI and join our community-backed growth journey.