NotesAI Engineer

The learning path

Roadmap

Sixteen weeks, nine subject tracks, five projects. Math and classical ML first and in parallel, then deep learning, then everything that makes an AI engineer.

W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16 study 1 Math 2 ML 3 DL 4 LLM internals 5 6 RAG 7 Agents 8 9 build P0 ML + NN from scratch P1 GPT P2 P3 RAG P4 Agent + prod Polish
Fig. 1. Sixteen weeks at a glance: what you study (top) and what you build (bottom). Math and ML share weeks 1–4. Bars 5, 8 and 9 are the modern LLM stack, evals and production, and the interview drill.

Why math and ML run together

Studying math alone is dry and studying ML without it is confusing. So each week pairs the two: the math note gives you the tool, and the ML note of the same week uses it immediately. Vectors and dot products appear on Monday, linear regression uses them on Tuesday. Derivatives arrive in week 2 and gradient descent spends them the same week. By week 5 the neural network note reads easily, because every symbol in it has already done a job.

How to study each day

The order matters twice over. Read the notes in the order the week-by-week table below gives, because each note only uses words that earlier notes explained; and meet each idea three times, in three forms, because that is what makes it stick. If a word stops you, hover over it (dotted underline) or open the glossary; if a line of code stops you, click the np. call to open the NumPy reference.

  1. Watch or read the source for the topic (the lecture or chapter linked inside the note).
  2. Read our note on the same topic. Redraw its main diagram on paper from memory.
  3. Do the practice. Hand problems on paper first, then the coding drill typed by hand. Do not paste.
  4. Say the interview answer out loud without looking. If you get stuck, that is the part to reread.
  5. Answer the practice questions at the end of the note before opening them.
DayWhat to do
Monday to FridayOne or two notes a day with their practice drills. In weeks 1–4, alternate a math note and an ML note.
SaturdayProject work only. Build the milestone listed for that week on the practice page.
SundayRevision cards and practice questions for the whole week, plus a timed drill from the practice page. Then rest.

Week by week

Each row is one week, and the notes are listed in the order to read them. In weeks 1–4 that order alternates between the ML and the math track on purpose: the ML note raises a question and the next math note answers it, so no note uses a word that an earlier one has not explained. The full practice for the week is on the practice page.

WeekThemeNotes
W1 Vectors, matrices and linear regression
  1. ML What machine learning is: data, model, loss, optimizer
  2. Math Functions, exponents and logarithms
  3. Math Vectors and the dot product
  4. ML Linear regression
  5. Math Matrices, matrix multiplication and shapes
  6. Math Norms, distance and cosine similarity
  7. ML Feature scaling, normalization and encoding
W2 Derivatives, gradients and gradient descent
  1. Math Derivatives: slope and rate of change
  2. ML Loss functions: MSE, MAE and log loss
  3. Math Partial derivatives and the gradient
  4. ML Gradient descent: batch, stochastic and mini-batch
  5. Math The chain rule
  6. ML Overfitting, underfitting and the bias–variance trade-off
W3 Probability, logistic regression and evaluation
  1. Math Mean, median, variance and standard deviation
  2. Math Probability, conditional probability and Bayes’ theorem
  3. ML Logistic regression and the sigmoid
  4. Math Distributions, expectation and the normal distribution
  5. ML Evaluation metrics: accuracy, precision, recall, F1, ROC-AUC
  6. Math Maximum likelihood: where loss functions come from
  7. ML Train, validation, test and cross-validation
  8. ML Regularization: L1, L2 and early stopping
W4 Entropy, regularization, similarity models and trees
  1. Math Entropy, cross-entropy and KL divergence
  2. ML kNN and k-means: learning from similarity
  3. Math Eigenvectors and SVD, only as much as PCA and LoRA need
  4. ML PCA and dimensionality reduction
  5. ML Decision trees, random forests and gradient boosting
  6. ML SVM and Naive Bayes at interview depth
  7. ML Class imbalance, data leakage and other traps
  8. Math Math you can skip, and what to say if it comes up
W5 Neurons, backpropagation and softmax
  1. DL What is a neural network?
  2. DL Backpropagation: how networks learn
  3. DL Logits, softmax and cross-entropy
  4. DL Build a neural network in NumPy, end to end
W6 Optimizers, PyTorch and regularization
  1. DL Optimizers: SGD with momentum, Adam, AdamW and the learning rate
  2. DL PyTorch essentials: tensors, autograd, the training loop
  3. DL Dropout, BatchNorm, LayerNorm and weight decay
  4. DL CNNs, RNNs and LSTMs: what they are and why attention replaced them
W7 Tokens, embeddings and the attention idea
  1. LLM internals What is an LLM? Pretraining, fine-tuning and the big picture
  2. LLM internals Tokenization and byte-pair encoding (BPE)
  3. LLM internals Token embeddings: turning IDs into vectors
  4. LLM internals Positional encoding: telling the model about word order
  5. LLM internals Preparing data: sliding windows and next-token prediction
  6. LLM internals Self-attention, the simple version
W8 Full attention and the transformer block
  1. LLM internals Scaled dot-product attention: queries, keys and values
  2. LLM internals Causal masking and attention dropout
  3. LLM internals Multi-head attention
  4. LLM internals LayerNorm, GELU, feed-forward layers and residual connections
  5. LLM internals The transformer block and the full GPT architecture
  6. LLM internals Counting parameters and weight tying
W9 Training, decoding and fine-tuning
  1. LLM internals Pretraining: the training loop, cross-entropy and perplexity
  2. LLM internals Decoding: greedy, temperature, top-k and top-p
  3. LLM internals Loading pretrained GPT-2 weights
  4. LLM internals Fine-tuning for classification
  5. LLM internals Instruction fine-tuning
  6. LLM internals Encoder, decoder, encoder–decoder: BERT vs GPT vs T5
W10 Modern architecture, LoRA and quantization
  1. Modern From GPT-2 to modern LLMs: pre-norm, RMSNorm, SwiGLU, grouped-query attention
  2. Modern Rotary position embeddings (RoPE)
  3. Modern The KV cache: why generation is fast
  4. Modern Quantization: 16-bit, 8-bit, 4-bit and GGUF
  5. Modern LoRA, QLoRA and parameter-efficient fine-tuning
  6. Modern Teaching preferences: RLHF and DPO
  7. Modern Context windows and reasoning models
  8. Modern Prompting vs RAG vs fine-tuning: how to choose
W11 Embeddings, vector search and retrieval
  1. RAG Text embeddings and semantic similarity
  2. RAG Vector databases and approximate nearest neighbour search (HNSW, IVF)
  3. RAG Chunking strategies
  4. RAG Retrieval: BM25, dense, hybrid search and rank fusion
  5. RAG Reranking with cross-encoders
W12 The RAG pipeline and its evaluation
  1. RAG The RAG pipeline end to end
  2. RAG Advanced RAG: query rewriting, HyDE, multi-hop, GraphRAG, agentic RAG
  3. RAG Evaluating RAG: recall@k, MRR, nDCG, faithfulness
  4. RAG RAG failure modes and how to debug them
W13 Prompting, tools and the agent loop
  1. Agents Prompt engineering that survives production
  2. Agents Context engineering: what goes in the window
  3. Agents Structured output and function (tool) calling
  4. Agents The agent loop and ReAct
  5. Agents Workflows vs agents: routing, orchestrator–workers, evaluator–optimizer
W14 MCP, memory and security
  1. Agents Model Context Protocol (MCP)
  2. Agents Agent memory: short-term, long-term, compaction
  3. Agents Multi-agent systems: when they help and when they hurt
  4. Agents Prompt injection and agent security
W15 Evals, cost and serving
  1. Prod Evals: golden sets, metrics and regression testing
  2. Prod LLM-as-judge: how to trust a model grading a model
  3. Prod Hallucination: causes, detection, mitigation
  4. Prod Guardrails and safety layers
  5. Prod Cost and latency: caching, batching, streaming, model routing
  6. Prod Serving LLMs: time to first token, throughput, continuous batching
  7. Prod Observability and tracing
W16 Interview drill
  1. Interview A framework for LLM system design rounds
  2. Interview Design: a customer support assistant
  3. Interview Design: enterprise document search at scale
  4. Interview Design: a coding assistant
  5. Interview The ML coding round: attention, BPE and sampling from memory
  6. Interview Question bank: math, ML and DL fundamentals
  7. Interview Question bank: applied AI engineering
  8. Interview Talking about your projects

The nine tracks

1

Math for AI interviews

Weeks 1–4, alongside ML

Goal. Read any formula in these notes without fear, and do the small calculations interviewers ask for by hand.

Scope. Only the math that AI engineer interviews ask or that a later note needs: vectors, matrices, slopes, the chain rule, probability and entropy. No proofs, no integrals, no theory for its own sake. The last note lists what you may skip.

Study from.

  • 3Blue1Brown, “Essence of linear algebra” and “Essence of calculus” (YouTube)
  • Khan Academy: linear algebra, differential calculus, statistics and probability (for practice problems)
  • Deisenroth, Faisal and Ong, Mathematics for Machine Learning (free PDF), chapters 2–6, selected sections
  • Seeing Theory (Brown University), a visual introduction to probability

Notes.

  1. Functions, exponents and logarithms W1 ready
  2. Vectors and the dot product W1 ready
  3. Matrices, matrix multiplication and shapes W1 ready
  4. Norms, distance and cosine similarity W1 ready
  5. Derivatives: slope and rate of change W2 to be written
  6. Partial derivatives and the gradient W2 to be written
  7. The chain rule W2 to be written
  8. Mean, median, variance and standard deviation W3 to be written
  9. Probability, conditional probability and Bayes’ theorem W3 to be written
  10. Distributions, expectation and the normal distribution W3 to be written
  11. Maximum likelihood: where loss functions come from W3 to be written
  12. Entropy, cross-entropy and KL divergence W4 to be written
  13. Eigenvectors and SVD, only as much as PCA and LoRA need W4 to be written
  14. Math you can skip, and what to say if it comes up W4 to be written
2

Classical machine learning

Weeks 1–4, alongside Math

Goal. Explain how a model learns from data, train and evaluate one properly, and name the right algorithm for a problem.

Scope. The algorithms and habits interviews actually test: regression, classification, loss, gradient descent, overfitting, regularization, metrics, data splits, plus kNN, k-means, PCA and tree models at the depth they are asked. Each ML note is paired with the math note it needs.

Build. Project 0: an ML library from scratch, plus a tabular case study

Study from.

  • Andrew Ng, Machine Learning Specialization (Coursera), courses 1–2, watched at 1.5x
  • StatQuest with Josh Starmer (YouTube), one video per algorithm
  • Raschka, Machine Learning with PyTorch and Scikit-Learn, chapters 2–6 and 10
  • James, Witten, Hastie and Tibshirani, An Introduction to Statistical Learning (free PDF), chapters 2–5
  • Google, Machine Learning Crash Course (for quick checks)

Notes.

  1. What machine learning is: data, model, loss, optimizer W1 ready
  2. Linear regression W1 ready
  3. Feature scaling, normalization and encoding W1 ready
  4. Loss functions: MSE, MAE and log loss W2 to be written
  5. Gradient descent: batch, stochastic and mini-batch W2 to be written
  6. Overfitting, underfitting and the bias–variance trade-off W2 to be written
  7. Logistic regression and the sigmoid W3 to be written
  8. Evaluation metrics: accuracy, precision, recall, F1, ROC-AUC W3 to be written
  9. Train, validation, test and cross-validation W3 to be written
  10. Regularization: L1, L2 and early stopping W3 to be written
  11. kNN and k-means: learning from similarity W4 to be written
  12. PCA and dimensionality reduction W4 to be written
  13. Decision trees, random forests and gradient boosting W4 to be written
  14. SVM and Naive Bayes at interview depth W4 to be written
  15. Class imbalance, data leakage and other traps W4 to be written
3

Deep learning foundations

Weeks 5–6

Goal. Understand how any neural network learns, build one in NumPy, then write a PyTorch training loop from memory.

Scope. Neurons to networks, backpropagation, optimizers, softmax, PyTorch, the regularization tricks specific to deep nets, and a short bridge through CNNs and RNNs to attention. Computer vision and speech architectures are covered only as far as interviews ask.

Build. Project 0 continued: a neural network in NumPy, then the same network in PyTorch

Study from.

  • 3Blue1Brown, “Neural Networks” series (chapters 1–4)
  • Andrej Karpathy, “Zero to Hero”: the micrograd video
  • Michael Nielsen, Neural Networks and Deep Learning (free online), chapters 1–3
  • Raschka, Build a Large Language Model (From Scratch), Appendix A: Introduction to PyTorch

Notes.

  1. What is a neural network? W5 ready
  2. Backpropagation: how networks learn W5 to be written
  3. Logits, softmax and cross-entropy W5 to be written
  4. Build a neural network in NumPy, end to end W5 to be written
  5. Optimizers: SGD with momentum, Adam, AdamW and the learning rate W6 to be written
  6. PyTorch essentials: tensors, autograd, the training loop W6 to be written
  7. Dropout, BatchNorm, LayerNorm and weight decay W6 to be written
  8. CNNs, RNNs and LSTMs: what they are and why attention replaced them W6 to be written
4

LLM internals: build GPT from scratch

Weeks 7–9

Goal. Explain and code every part of a GPT-style model: text in, next token out.

Scope. The full GPT-2 architecture and training process as in the Vizuara course and the Raschka book, plus the encoder-decoder family for comparison.

Build. Project 1: GPT from scratch

Study from.

  • Vizuara, “Building LLMs from Scratch” (the full playlist, at 1.5–2x)
  • Raschka, Build a Large Language Model (From Scratch), chapters 1–7
  • Vaswani et al., “Attention Is All You Need” (2017)
  • Jay Alammar, “The Illustrated Transformer” and “The Illustrated GPT-2”
  • Karpathy, “Let’s build GPT: from scratch, in code, spelled out”

Notes.

  1. What is an LLM? Pretraining, fine-tuning and the big picture W7 to be written
  2. Tokenization and byte-pair encoding (BPE) W7 to be written
  3. Token embeddings: turning IDs into vectors W7 to be written
  4. Positional encoding: telling the model about word order W7 to be written
  5. Preparing data: sliding windows and next-token prediction W7 to be written
  6. Self-attention, the simple version W7 to be written
  7. Scaled dot-product attention: queries, keys and values W8 to be written
  8. Causal masking and attention dropout W8 to be written
  9. Multi-head attention W8 to be written
  10. LayerNorm, GELU, feed-forward layers and residual connections W8 to be written
  11. The transformer block and the full GPT architecture W8 to be written
  12. Counting parameters and weight tying W8 to be written
  13. Pretraining: the training loop, cross-entropy and perplexity W9 to be written
  14. Decoding: greedy, temperature, top-k and top-p W9 to be written
  15. Loading pretrained GPT-2 weights W9 to be written
  16. Fine-tuning for classification W9 to be written
  17. Instruction fine-tuning W9 to be written
  18. Encoder, decoder, encoder–decoder: BERT vs GPT vs T5 W9 to be written
5

The modern LLM stack

Week 10

Goal. Know what changed between GPT-2 and today’s models, and how models are adapted and shrunk cheaply.

Scope. Architecture changes, RoPE, the KV cache, quantization, LoRA and QLoRA, preference tuning, and how to choose between prompting, RAG and fine-tuning.

Build. Project 2: QLoRA fine-tune, quantize, run locally

Study from.

  • Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models” (2021)
  • Dettmers et al., “QLoRA” (2023)
  • Su et al., “RoFormer: Rotary Position Embedding” (2021)
  • Raschka, Build a Large Language Model (From Scratch), Appendix E: LoRA
  • Hugging Face PEFT and Transformers documentation

Notes.

  1. From GPT-2 to modern LLMs: pre-norm, RMSNorm, SwiGLU, grouped-query attention W10 to be written
  2. Rotary position embeddings (RoPE) W10 to be written
  3. The KV cache: why generation is fast W10 to be written
  4. Quantization: 16-bit, 8-bit, 4-bit and GGUF W10 to be written
  5. LoRA, QLoRA and parameter-efficient fine-tuning W10 to be written
  6. Teaching preferences: RLHF and DPO W10 to be written
  7. Context windows and reasoning models W10 to be written
  8. Prompting vs RAG vs fine-tuning: how to choose W10 to be written
6

Retrieval-augmented generation (RAG)

Weeks 11–12

Goal. Design, build and measure a RAG system, and explain every trade-off in it.

Scope. Embeddings, vector search, chunking, hybrid retrieval, reranking, the full pipeline, advanced variants, evaluation and debugging.

Build. Project 3: production RAG with evals

Study from.

  • Chip Huyen, AI Engineering (2025), the RAG and agents chapter
  • Jay Alammar and Maarten Grootendorst, Hands-On Large Language Models
  • Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (2020)
  • Malkov and Yashunin, “HNSW” (2016)
  • Anthropic, “Introducing Contextual Retrieval” (2024)

Notes.

  1. Text embeddings and semantic similarity W11 to be written
  2. Vector databases and approximate nearest neighbour search (HNSW, IVF) W11 to be written
  3. Chunking strategies W11 to be written
  4. Retrieval: BM25, dense, hybrid search and rank fusion W11 to be written
  5. Reranking with cross-encoders W11 to be written
  6. The RAG pipeline end to end W12 to be written
  7. Advanced RAG: query rewriting, HyDE, multi-hop, GraphRAG, agentic RAG W12 to be written
  8. Evaluating RAG: recall@k, MRR, nDCG, faithfulness W12 to be written
  9. RAG failure modes and how to debug them W12 to be written
7

Prompting, tools and agents

Weeks 13–14

Goal. Build reliable LLM features: good context, structured output, tool use and agent loops.

Scope. Prompt and context engineering, structured output, the agent loop, agent patterns, MCP, memory, multi-agent systems and security.

Build. Project 4: agent with your own MCP server

Study from.

  • Anthropic, “Building Effective Agents” (2024)
  • Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” (2022)
  • Model Context Protocol specification and documentation
  • Anthropic and OpenAI prompt engineering guides
  • Chip Huyen, AI Engineering (2025), the prompt engineering chapter

Notes.

  1. Prompt engineering that survives production W13 to be written
  2. Context engineering: what goes in the window W13 to be written
  3. Structured output and function (tool) calling W13 to be written
  4. The agent loop and ReAct W13 to be written
  5. Workflows vs agents: routing, orchestrator–workers, evaluator–optimizer W13 to be written
  6. Model Context Protocol (MCP) W14 to be written
  7. Agent memory: short-term, long-term, compaction W14 to be written
  8. Multi-agent systems: when they help and when they hurt W14 to be written
  9. Prompt injection and agent security W14 to be written
8

Evals and production

Week 15

Goal. Prove an LLM feature works, then run it cheaply, quickly and safely.

Scope. Evaluation, LLM-as-judge, hallucination, guardrails, cost, latency, serving and observability.

Build. Project 4 continued: evals, guardrails, cost and latency report

Study from.

  • Chip Huyen, AI Engineering (2025), the evaluation and inference optimization chapters
  • Hamel Husain, “Your AI Product Needs Evals”
  • vLLM documentation (continuous batching, PagedAttention)

Notes.

  1. Evals: golden sets, metrics and regression testing W15 to be written
  2. LLM-as-judge: how to trust a model grading a model W15 to be written
  3. Hallucination: causes, detection, mitigation W15 to be written
  4. Guardrails and safety layers W15 to be written
  5. Cost and latency: caching, batching, streaming, model routing W15 to be written
  6. Serving LLMs: time to first token, throughput, continuous batching W15 to be written
  7. Observability and tracing W15 to be written
9

LLM system design and interview drill

Week 16, then every Sunday while applying

Goal. Turn everything into spoken, structured answers under time pressure.

Scope. A framework for design rounds, three worked designs, the ML coding round, question banks and project storytelling.

Build. Polish all projects: READMEs, demos, resume bullets

Study from.

  • Our own notes, tracks 1–8
  • Alex Xu, System Design Interview (for the classic distributed-systems parts)
  • Chip Huyen, AI Engineering (2025), the architecture chapter

Notes.

  1. A framework for LLM system design rounds W16 to be written
  2. Design: a customer support assistant W16 to be written
  3. Design: enterprise document search at scale W16 to be written
  4. Design: a coding assistant W16 to be written
  5. The ML coding round: attention, BPE and sampling from memory W16 to be written
  6. Question bank: math, ML and DL fundamentals W16 to be written
  7. Question bank: applied AI engineering W16 to be written
  8. Talking about your projects W16 to be written

How you know a track is done

  • You can redraw every main diagram of the track on a blank page.
  • You can give each note’s interview answer in under two minutes, without notes.
  • You can do the week’s hand problems without looking anything up.
  • You can answer at least four in five practice questions correctly.
  • The project milestone for the track runs, and its README explains what you measured.