Notes

Only the math that AI engineer interviews ask or that a later note needs: vectors, matrices, slopes, the chain rule, probability and entropy. No proofs, no integrals, no theory for its own sake. The last note lists what you may skip.

1. Functions, exponents and logarithms W1 ready
2. Vectors and the dot product W1 ready
3. Matrices, matrix multiplication and shapes W1 ready
4. Norms, distance and cosine similarity W1 ready
5. Derivatives: slope and rate of change W2 to be written
6. Partial derivatives and the gradient W2 to be written
7. The chain rule W2 to be written
8. Mean, median, variance and standard deviation W3 to be written
9. Probability, conditional probability and Bayes’ theorem W3 to be written
10. Distributions, expectation and the normal distribution W3 to be written
11. Maximum likelihood: where loss functions come from W3 to be written
12. Entropy, cross-entropy and KL divergence W4 to be written
13. Eigenvectors and SVD, only as much as PCA and LoRA need W4 to be written
14. Math you can skip, and what to say if it comes up W4 to be written

The algorithms and habits interviews actually test: regression, classification, loss, gradient descent, overfitting, regularization, metrics, data splits, plus kNN, k-means, PCA and tree models at the depth they are asked. Each ML note is paired with the math note it needs.

1. What machine learning is: data, model, loss, optimizer W1 ready
2. Linear regression W1 ready
3. Feature scaling, normalization and encoding W1 ready
4. Loss functions: MSE, MAE and log loss W2 to be written
5. Gradient descent: batch, stochastic and mini-batch W2 to be written
6. Overfitting, underfitting and the bias–variance trade-off W2 to be written
7. Logistic regression and the sigmoid W3 to be written
8. Evaluation metrics: accuracy, precision, recall, F1, ROC-AUC W3 to be written
9. Train, validation, test and cross-validation W3 to be written
10. Regularization: L1, L2 and early stopping W3 to be written
11. kNN and k-means: learning from similarity W4 to be written
12. PCA and dimensionality reduction W4 to be written
13. Decision trees, random forests and gradient boosting W4 to be written
14. SVM and Naive Bayes at interview depth W4 to be written
15. Class imbalance, data leakage and other traps W4 to be written

Neurons to networks, backpropagation, optimizers, softmax, PyTorch, the regularization tricks specific to deep nets, and a short bridge through CNNs and RNNs to attention. Computer vision and speech architectures are covered only as far as interviews ask.

1. What is a neural network? W5 ready
2. Backpropagation: how networks learn W5 to be written
3. Logits, softmax and cross-entropy W5 to be written
4. Build a neural network in NumPy, end to end W5 to be written
5. Optimizers: SGD with momentum, Adam, AdamW and the learning rate W6 to be written
6. PyTorch essentials: tensors, autograd, the training loop W6 to be written
7. Dropout, BatchNorm, LayerNorm and weight decay W6 to be written
8. CNNs, RNNs and LSTMs: what they are and why attention replaced them W6 to be written

The full GPT-2 architecture and training process as in the Vizuara course and the Raschka book, plus the encoder-decoder family for comparison.

1. What is an LLM? Pretraining, fine-tuning and the big picture W7 to be written
2. Tokenization and byte-pair encoding (BPE) W7 to be written
3. Token embeddings: turning IDs into vectors W7 to be written
4. Positional encoding: telling the model about word order W7 to be written
5. Preparing data: sliding windows and next-token prediction W7 to be written
6. Self-attention, the simple version W7 to be written
7. Scaled dot-product attention: queries, keys and values W8 to be written
8. Causal masking and attention dropout W8 to be written
9. Multi-head attention W8 to be written
10. LayerNorm, GELU, feed-forward layers and residual connections W8 to be written
11. The transformer block and the full GPT architecture W8 to be written
12. Counting parameters and weight tying W8 to be written
13. Pretraining: the training loop, cross-entropy and perplexity W9 to be written
14. Decoding: greedy, temperature, top-k and top-p W9 to be written
15. Loading pretrained GPT-2 weights W9 to be written
16. Fine-tuning for classification W9 to be written
17. Instruction fine-tuning W9 to be written
18. Encoder, decoder, encoder–decoder: BERT vs GPT vs T5 W9 to be written

Architecture changes, RoPE, the KV cache, quantization, LoRA and QLoRA, preference tuning, and how to choose between prompting, RAG and fine-tuning.

1. From GPT-2 to modern LLMs: pre-norm, RMSNorm, SwiGLU, grouped-query attention W10 to be written
2. Rotary position embeddings (RoPE) W10 to be written
3. The KV cache: why generation is fast W10 to be written
4. Quantization: 16-bit, 8-bit, 4-bit and GGUF W10 to be written
5. LoRA, QLoRA and parameter-efficient fine-tuning W10 to be written
6. Teaching preferences: RLHF and DPO W10 to be written
7. Context windows and reasoning models W10 to be written
8. Prompting vs RAG vs fine-tuning: how to choose W10 to be written

Embeddings, vector search, chunking, hybrid retrieval, reranking, the full pipeline, advanced variants, evaluation and debugging.

1. Text embeddings and semantic similarity W11 to be written
2. Vector databases and approximate nearest neighbour search (HNSW, IVF) W11 to be written
3. Chunking strategies W11 to be written
4. Retrieval: BM25, dense, hybrid search and rank fusion W11 to be written
5. Reranking with cross-encoders W11 to be written
6. The RAG pipeline end to end W12 to be written
7. Advanced RAG: query rewriting, HyDE, multi-hop, GraphRAG, agentic RAG W12 to be written
8. Evaluating RAG: recall@k, MRR, nDCG, faithfulness W12 to be written
9. RAG failure modes and how to debug them W12 to be written

Prompt and context engineering, structured output, the agent loop, agent patterns, MCP, memory, multi-agent systems and security.

1. Prompt engineering that survives production W13 to be written
2. Context engineering: what goes in the window W13 to be written
3. Structured output and function (tool) calling W13 to be written
4. The agent loop and ReAct W13 to be written
5. Workflows vs agents: routing, orchestrator–workers, evaluator–optimizer W13 to be written
6. Model Context Protocol (MCP) W14 to be written
7. Agent memory: short-term, long-term, compaction W14 to be written
8. Multi-agent systems: when they help and when they hurt W14 to be written
9. Prompt injection and agent security W14 to be written

Evaluation, LLM-as-judge, hallucination, guardrails, cost, latency, serving and observability.

1. Evals: golden sets, metrics and regression testing W15 to be written
2. LLM-as-judge: how to trust a model grading a model W15 to be written
3. Hallucination: causes, detection, mitigation W15 to be written
4. Guardrails and safety layers W15 to be written
5. Cost and latency: caching, batching, streaming, model routing W15 to be written
6. Serving LLMs: time to first token, throughput, continuous batching W15 to be written
7. Observability and tracing W15 to be written

A framework for design rounds, three worked designs, the ML coding round, question banks and project storytelling.

1. A framework for LLM system design rounds W16 to be written
2. Design: a customer support assistant W16 to be written
3. Design: enterprise document search at scale W16 to be written
4. Design: a coding assistant W16 to be written
5. The ML coding round: attention, BPE and sampling from memory W16 to be written
6. Question bank: math, ML and DL fundamentals W16 to be written
7. Question bank: applied AI engineering W16 to be written
8. Talking about your projects W16 to be written