NotesAI Engineer

Reference · 60 entries

NumPy, as the notes use it

Every NumPy function that appears in a code block anywhere on this site has an entry here. Click any np. call inside a code block and you land on it. Each entry says what the function does in plain words, shows a short example with its real output, lists the mistakes people make with it, and gives one sentence you could say about it in an interview.

Arrays: making them

import numpy as np · np.array · np.arange · np.linspace · np.zeros · np.ones · np.full · np.zeros_like, np.ones_like, np.full_like · np.eye · np.diag

Shape: asking and changing

.shape, .ndim, .size, .dtype · .reshape · .T (transpose) · v[:, None] and np.newaxis · np.stack, np.vstack, np.column_stack, np.concatenate · .ravel and .flatten · .astype

Picking elements: indexing, slicing, masks

Indexing and slicing · Boolean masks · Index with an array of positions · np.where · np.unravel_index

Math on every element

Element-wise arithmetic and broadcasting · np.exp · np.log · np.sqrt · np.abs · np.round · np.maximum and np.minimum · np.clip · np.sin, np.cos, np.arccos, np.degrees · np.e, np.pi, np.inf, np.nan

Reductions: one number from many

np.sum and .sum() · np.mean and .mean() · np.median and np.percentile · np.std, np.var and .std() · np.max, np.min, .max(), .min() · np.argmax, np.argmin, .argmax() · np.sort and np.argsort · np.prod

Linear algebra: dot, matmul, norm, solve

np.dot · @ and np.matmul · np.linalg.norm · np.linalg.inv · np.linalg.solve · np.linalg.lstsq · np.polyfit and np.polyval · np.linalg.matrix_rank · np.linalg.LinAlgError · np.linalg.eig, np.linalg.eigh, np.linalg.svd

Random numbers

np.random.default_rng · rng.normal · rng.uniform · rng.integers · rng.permutation and rng.shuffle · rng.choice

Checking and printing

np.errstate · np.isclose and np.allclose · np.set_printoptions · .fill and in-place updates

Arrays: making them

NumPy is the library every piece of ML code in these notes uses. Its one data type is the array: a grid of numbers of one type, stored tightly in memory, with every math operation written once for the whole grid instead of in a loop. These are the ways to make one.

§ import numpy as np

Loads the library under the short name np. Every example on this page assumes this line has run.

Call it as import numpy as np

import numpy as np
print(np.__version__)
output2.2.6
Watch out
  • Nobody writes numpy.array; the np alias is universal, and interviewers write it too.

Official documentation ↗

§ np.array

Turns a Python list (or list of lists) into an array. A list of numbers gives a 1-D array; a list of equal-length lists gives a 2-D array, one inner list per row.

Call it as np.array(object, dtype=None)

import numpy as np
v = np.array([1, 2, 3])
M = np.array([[1, 2, 3],
              [4, 5, 6]])
print(v, v.shape, v.dtype)
print(M)
print(M.shape)
print(np.array([1, 2, 3], dtype=float))
output[1 2 3] (3,) int64
[[1 2 3]
 [4 5 6]]
(2, 3)
[1. 2. 3.]
Watch out
  • Mixing an int and a float in the list makes the whole array float; mixing in a string makes everything a string.
  • np.array([1, 2, 3]) is an array of shape (3,), not (3, 1) and not (1, 3). The matrices note explains why that matters.

Official documentation ↗

§ np.arange

Numbers from a start up to but not including a stop, in steps. Like Python's range, but it returns an array and allows fractional steps.

Call it as np.arange(start, stop, step)

import numpy as np
print(np.arange(5))
print(np.arange(2, 10, 2))
print(np.arange(0, 1, 0.25))
output[0 1 2 3 4]
[2 4 6 8]
[0.   0.25 0.5  0.75]
Watch out
  • The stop value is never included.
  • With fractional steps the number of elements can be off by one because of rounding; use np.linspace when you care about the count.

Official documentation ↗

§ np.linspace

Exactly num numbers spread evenly from start to stop, both included. The right tool for the x-axis of a plot.

Call it as np.linspace(start, stop, num=50)

import numpy as np
print(np.linspace(0, 1, 5))
print(np.linspace(-2, 2, 9))
output[0.   0.25 0.5  0.75 1.  ]
[-2.  -1.5 -1.  -0.5  0.   0.5  1.   1.5  2. ]
Watch out
  • Unlike arange, the stop value is included.

Official documentation ↗

§ np.zeros

An array of the given shape filled with 0.0. The usual way to make a place to accumulate results into.

Call it as np.zeros(shape, dtype=float)

import numpy as np
print(np.zeros(3))
print(np.zeros((2, 3)))
print(np.zeros(3, dtype=int))
output[0. 0. 0.]
[[0. 0. 0.]
 [0. 0. 0.]]
[0 0 0]
Watch out
  • The shape of a 2-D array is passed as one tuple, np.zeros((2, 3)), not np.zeros(2, 3).

Official documentation ↗

§ np.ones

An array of the given shape filled with 1.0. Often used to add a column of ones so that the bias term can live inside the weight vector.

Call it as np.ones(shape, dtype=float)

import numpy as np
X = np.array([[1.0], [1.5], [2.0]])
X1 = np.column_stack([np.ones(3), X])   # bias column first
print(X1)
output[[1.  1. ]
 [1.  1.5]
 [1.  2. ]]
Watch out
  • Same tuple rule as np.zeros.

Official documentation ↗

§ np.full

An array of the given shape filled with any value you choose.

Call it as np.full(shape, fill_value)

import numpy as np
print(np.full((2, 2), 7))
print(np.full(3, np.nan))
output[[7 7]
 [7 7]]
[nan nan nan]
Watch out
  • np.full(3, 7) is integer; np.full(3, 7.0) is float.

Official documentation ↗

§ np.zeros_like, np.ones_like, np.full_like

The same as zeros, ones and full, but copying the shape and type from an existing array instead of spelling them out.

Call it as np.zeros_like(a) np.ones_like(a) np.full_like(a, value)

import numpy as np
y = np.array([200.0, 300.0, 380.0])
print(np.zeros_like(y))
print(np.ones_like(y))
print(np.full_like(y, y.mean()))   # a 'predict the average' baseline
output[0. 0. 0.]
[1. 1. 1.]
[293.33333333 293.33333333 293.33333333]
Watch out
  • full_like keeps the dtype of the template, so np.full_like(np.array([1, 2]), 2.5) silently gives [2, 2].

Official documentation ↗

§ np.eye

The identity matrix: ones on the diagonal, zeros elsewhere. Multiplying by it changes nothing.

Call it as np.eye(n)

import numpy as np
I = np.eye(3)
A = np.array([[2.0, 1.0, 0.0], [0.0, 1.0, 3.0], [4.0, 0.0, 1.0]])
print(I)
print(np.allclose(A @ I, A))
output[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]
True
Watch out
  • np.eye(3) is float, which is what you want for linear algebra.

Official documentation ↗

§ np.diag

Two jobs: given a 1-D array it builds a square matrix with those numbers on the diagonal; given a 2-D array it pulls the diagonal out.

Call it as np.diag(v) or np.diag(M)

import numpy as np
print(np.diag([1, 2, 3]))
M = np.arange(9).reshape(3, 3)
print(M)
print(np.diag(M))
output[[1 0 0]
 [0 2 0]
 [0 0 3]]
[[0 1 2]
 [3 4 5]
 [6 7 8]]
[0 4 8]
Watch out
  • Remember it goes both ways; which way depends only on whether you pass 1-D or 2-D.

Official documentation ↗

Shape: asking and changing

Most NumPy bugs are shape bugs. These are the tools for asking what shape an array has and changing it without changing the numbers.

§ .shape, .ndim, .size, .dtype

Four attributes to read, not call. shape is a tuple with one entry per axis (rows, columns, ...). ndim is how many axes. size is the total count of numbers. dtype is the number type.

Call it as a.shape a.ndim a.size a.dtype

import numpy as np
X = np.array([[1.0, 1.5, 2.0], [3.0, 4.0, 5.0]])
print(X.shape)
print(X.ndim, X.size, X.dtype)
v = np.array([1, 2, 3])
print(v.shape, v.ndim)
output(2, 3)
2 6 float64
(3,) 1
Watch out
  • shape is a tuple: (3,) with the comma is a 1-D array of three numbers; (3, 1) is a column; (1, 3) is a row.
  • They are attributes, so X.shape, not X.shape().

Official documentation ↗

§ .reshape

Rearranges the same numbers into a new shape, reading them row by row. One dimension may be -1, meaning 'work this one out'.

Call it as a.reshape(new_shape)

import numpy as np
a = np.arange(6)
print(a.reshape(2, 3))
print(a.reshape(3, -1))        # -1: NumPy computes 2
print(a.reshape(-1, 1).shape)  # a column vector
output[[0 1 2]
 [3 4 5]]
[[0 1]
 [2 3]
 [4 5]]
(6, 1)
Watch out
  • The number of elements must not change: 6 numbers cannot become (4, 2).
  • reshape does not transpose. (2, 3) reshaped to (3, 2) is not the transpose; it re-reads the numbers row by row.

Official documentation ↗

§ .T (transpose)

Flips a matrix over its diagonal: rows become columns. Shape (m, n) becomes (n, m).

Call it as M.T

import numpy as np
X = np.array([[1, 2, 3],
              [4, 5, 6]])
print(X.T)
print(X.shape, '->', X.T.shape)
v = np.array([1, 2, 3])
print(v.T.shape)   # unchanged: 1-D arrays have nothing to flip
output[[1 4]
 [2 5]
 [3 6]]
(2, 3) -> (3, 2)
(3,)
Watch out
  • Transposing a 1-D array does nothing. To get a column, use v.reshape(-1, 1) or v[:, None].

Official documentation ↗

§ v[:, None] and np.newaxis

Adds a new axis of length 1. v[:, None] turns a 1-D array of length n into an (n, 1) column; v[None, :] into a (1, n) row. Needed to make broadcasting do what you mean.

Call it as v[:, None] v[None, :] v[:, np.newaxis]

import numpy as np
v = np.array([1, 2, 3])
print(v[:, None].shape, v[None, :].shape)
# every product v_i * v_j: an outer product
print(v[:, None] * v[None, :])
output(3, 1) (1, 3)
[[1 2 3]
 [2 4 6]
 [3 6 9]]
Watch out
  • None and np.newaxis are the same thing; None is shorter and what most code uses.
  • Scikit-learn wants features as (n_samples, n_features), so a single feature must be x[:, None], not x.

Official documentation ↗

§ np.stack, np.vstack, np.column_stack, np.concatenate

Ways to glue arrays together. concatenate joins along an existing axis. vstack stacks rows on top of each other. column_stack puts 1-D arrays side by side as columns. stack creates a new axis.

Call it as np.concatenate([a, b], axis=0) np.vstack([a, b]) np.column_stack([a, b]) np.stack([a, b])

import numpy as np
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
print(np.concatenate([a, b]))
print(np.vstack([a, b]))
print(np.column_stack([a, b]))
print(np.stack([a, b]).shape, np.stack([a, b], axis=1).shape)
output[1 2 3 4 5 6]
[[1 2 3]
 [4 5 6]]
[[1 4]
 [2 5]
 [3 6]]
(2, 3) (3, 2)
Watch out
  • concatenate needs the arrays to already have the axis you join on; to turn two 1-D arrays into two rows, use vstack or stack.
  • All of them take one list of arrays, not several arguments: np.vstack([a, b]), not np.vstack(a, b).

Official documentation ↗

§ .ravel and .flatten

Both return the array as one long 1-D array. ravel avoids copying when it can; flatten always copies.

Call it as a.ravel() a.flatten()

import numpy as np
M = np.array([[1, 2], [3, 4]])
print(M.ravel())
print(M.flatten())
output[1 2 3 4]
[1 2 3 4]
Watch out
  • Because ravel may share memory, changing the result can change the original. Use flatten when you intend to modify.

Official documentation ↗

§ .astype

Returns a copy with a different number type: ints to floats, floats to ints (which truncates), booleans to 0 and 1.

Call it as a.astype(dtype)

import numpy as np
mask = np.array([True, False, True])
print(mask.astype(int))
print(np.array([1, 2, 3]).astype(float))
print(np.array([2.9, -2.9]).astype(int))   # truncates toward zero
output[1 0 1]
[1. 2. 3.]
[ 2 -2]
Watch out
  • float to int truncates, it does not round. Use np.round first if you want rounding.
  • Integer arrays silently truncate division results in some operations; convert to float early.

Official documentation ↗

Picking elements: indexing, slicing, masks

How to pull out one number, a row, a column, or every element that passes a test.

§ Indexing and slicing

a[i] is one element, a[i, j] one element of a matrix, a[i] of a matrix is row i, a[:, j] is column j, a[1:3] is elements 1 and 2. Negative numbers count from the end.

Call it as a[i] M[i, j] M[i] M[:, j] a[start:stop:step]

import numpy as np
M = np.array([[10, 11, 12],
              [20, 21, 22],
              [30, 31, 32]])
print(M[0, 2])
print(M[1])        # row 1
print(M[:, 0])     # column 0
print(M[-1])       # last row
print(M[0:2, 1:])  # rows 0-1, columns 1 onward
x = np.arange(10)
print(x[::2], x[::-1])
output12
[20 21 22]
[10 20 30]
[30 31 32]
[[11 12]
 [21 22]]
[0 2 4 6 8] [9 8 7 6 5 4 3 2 1 0]
Watch out
  • Slices are views: changing M[0] changes M. Add .copy() if you need independence.
  • M[1] is a 1-D array of shape (3,), the row with its axis dropped; M[1:2] keeps shape (1, 3).

Official documentation ↗

§ Boolean masks

Compare an array to something and you get an array of True/False. Use that as an index to keep only the elements where it is True. Summing a mask counts the Trues.

Call it as a[a > 0] (a > 0).sum() (a > 0).mean()

import numpy as np
y_true = np.array([1, 0, 1, 1, 0])
y_pred = np.array([1, 0, 0, 1, 1])
print(y_true == y_pred)
print((y_true == y_pred).mean())      # accuracy
x = np.array([-2.0, 0.5, 3.0, -1.0])
print(x[x > 0])
# ReLU: keep positives, zero the rest
print(np.maximum(x, 0))
output[ True  True False  True False]
0.6
[0.5 3. ]
[0.  0.5 3.  0. ]
Watch out
  • Combine conditions with & and |, each in brackets: (x > 0) & (x < 2). The words and / or raise an error on arrays.

Official documentation ↗

§ Index with an array of positions

Pass a list or array of integer positions and you get those elements, in that order. This is how a shuffled split picks rows.

Call it as a[idx] M[idx] M[rows, cols]

import numpy as np
X = np.arange(10, 15)
idx = np.array([3, 0, 4])
print(X[idx])
rng = np.random.default_rng(0)
perm = rng.permutation(5)
print(perm, X[perm])
M = np.arange(9).reshape(3, 3)
print(M[[0, 2]])            # rows 0 and 2
print(M[[0, 1, 2], [2, 1, 0]])   # elements (0,2), (1,1), (2,0)
output[13 10 14]
[2 4 3 0 1] [12 14 13 10 11]
[[0 1 2]
 [6 7 8]]
[2 4 6]
Watch out
  • This returns a copy, unlike slicing.
  • M[[0, 1], [1, 2]] picks the pairs (0,1) and (1,2), not a 2×2 block; for a block use M[[0, 1]][:, [1, 2]] or np.ix_.

Official documentation ↗

§ np.where

Two uses. np.where(condition, a, b) picks from a where the condition is True and from b elsewhere. np.where(condition) alone returns the positions where it is True.

Call it as np.where(cond, a, b) np.where(cond)

import numpy as np
x = np.array([-2.0, 0.5, 3.0, -1.0])
print(np.where(x > 0, x, 0.0))        # ReLU again
print(np.where(x > 0))                # positions
print(np.where(x > 0)[0])
output[0.  0.5 3.  0. ]
(array([1, 2]),)
[1 2]
Watch out
  • The one-argument form returns a tuple with one array per axis, so you usually want [0] at the end.

Official documentation ↗

§ np.unravel_index

Turns a position in a flattened array back into (row, column). Used with argmin or argmax on a 2-D grid.

Call it as np.unravel_index(flat_index, shape)

import numpy as np
grid = np.array([[5.0, 2.0], [9.0, 1.0]])
flat = grid.argmin()
row, col = np.unravel_index(flat, grid.shape)
print(flat, int(row), int(col))
output3 1 1
Watch out
  • argmax/argmin on a matrix return a flat position unless you give an axis; this is the fix.

Official documentation ↗

Math on every element

These functions apply to every number in the array at once and return an array of the same shape. The maths notes use them on whole datasets in one line.

§ Element-wise arithmetic and broadcasting

+, -, *, / and ** work number by number on arrays of the same shape. If the shapes differ, NumPy stretches the smaller one along axes of length 1 (broadcasting). A number against an array is the simplest case.

Call it as a + b a * b a ** 2 a - a.mean(axis=0)

import numpy as np
x = np.array([1.0, 1.5, 2.0])
y = np.array([200.0, 300.0, 380.0])
print(180 * x + 23.33)                # a line, for every house at once
print(((180 * x + 23.33 - y) ** 2).mean())   # mean squared error
X = np.array([[1.0, 100.0], [2.0, 300.0], [3.0, 500.0]])
# (3,2) minus (2,): centre each column
print(X - X.mean(axis=0))
output[203.33 293.33 383.33]
22.222233333333335
[[  -1. -200.]
 [   0.    0.]
 [   1.  200.]]
Watch out
  • * is element-wise, not matrix multiplication. Use @ for matrices.
  • Shapes (3,) and (3, 1) broadcast to (3, 3), which is almost never what you wanted; check shapes when a result is suddenly square.

Official documentation ↗

§ np.exp

e raised to each element. Turns any number into a positive one, and small differences into big ratios. The building block of the sigmoid and softmax functions.

Call it as np.exp(x)

import numpy as np
print(np.exp(0), np.exp(1))
z = np.array([2.0, 1.0, 0.0])
print(np.exp(z))
print(np.exp(z) / np.exp(z).sum())    # scores to percentages (softmax)
print(np.exp(1000))                   # overflow -> inf, with a warning
output1.0 2.718281828459045
[7.3890561  2.71828183 1.        ]
[0.66524096 0.24472847 0.09003057]
inf
Watch out
  • np.exp(710) is already inf in 64-bit floats. Subtract the maximum first: np.exp(z - z.max()).
  • np.exp of an integer array returns floats, which is fine.

Official documentation ↗

§ np.log

The natural logarithm (base e) of each element. np.log(np.exp(x)) is x. It turns products into sums and squashes huge ranges, which is why losses are written with it.

Call it as np.log(x) np.log2(x) np.log10(x) np.log1p(x)

import numpy as np
print(np.log(1), np.log(np.e))
p = np.array([0.9, 0.8, 0.95])
print(np.log(p).sum(), np.log(p.prod()))   # sum of logs == log of product
# log loss: cheap when sure and right
print(-np.log(np.array([0.99, 0.5, 0.01])))
print(np.log(0))                            # -inf, with a warning
print(np.log2(1024), np.log10(1000))
output0.0 1.0
-0.3797973613595866 -0.3797973613595865
[0.01005034 0.69314718 4.60517019]
-inf
10.0 3.0
Watch out
  • np.log is the natural log, not base 10. Use np.log10 or np.log2 explicitly when you mean them.
  • np.log(0) is -inf and np.log of a negative is nan. Clip probabilities to [1e-12, 1] before taking logs in a loss.

Official documentation ↗

§ np.sqrt

Square root of each element. Appears in the length of a vector, the standard deviation and RMSE.

Call it as np.sqrt(x)

import numpy as np
v = np.array([3.0, 4.0])
print(np.sqrt((v ** 2).sum()))     # length of the vector
errors = np.array([-2.0, 1.0, 3.0])
print(np.sqrt((errors ** 2).mean()))   # RMSE
print(np.sqrt(-1))                 # nan, with a warning
output5.0
2.160246899469287
nan
Watch out
  • Square root of a negative is nan, not an error.

Official documentation ↗

§ np.abs

Absolute value of each element. Used in the L1 norm and in mean absolute error.

Call it as np.abs(x)

import numpy as np
errors = np.array([-2.0, 1.0, 3.0])
print(np.abs(errors))
print(np.abs(errors).mean())   # mean absolute error
print(np.abs(errors).sum())    # L1 norm
output[2. 1. 3.]
2.0
6.0

Official documentation ↗

§ np.round

Rounds each element to a number of decimals. Mostly for printing.

Call it as np.round(x, decimals=0)

import numpy as np
x = np.array([0.6652, 0.2447, 0.0900])
print(np.round(x, 2))
# rounds halves to the even neighbour
print(np.round(2.5), np.round(3.5))
output[0.67 0.24 0.09]
2.0 4.0
Watch out
  • Halves round to the nearest even number (2.5 → 2, 3.5 → 4), the banker's rule, unlike school rounding.

Official documentation ↗

§ np.maximum and np.minimum

Compares two arrays (or an array and a number) element by element and keeps the larger (or smaller) of each pair. np.maximum(x, 0) is ReLU.

Call it as np.maximum(a, b) np.minimum(a, b)

import numpy as np
x = np.array([-2.0, 0.5, 3.0, -1.0])
print(np.maximum(x, 0))
print(np.minimum(x, 1))
print(np.maximum(np.array([1, 5, 2]), np.array([3, 2, 2])))
output[0.  0.5 3.  0. ]
[-2.   0.5  1.  -1. ]
[3 5 2]
Watch out
  • np.maximum (two inputs, element-wise) is different from np.max (one input, reduces to its largest element).

Official documentation ↗

§ np.clip

Pushes every element into a range: anything below the minimum becomes the minimum, anything above the maximum becomes the maximum. Used to keep probabilities away from exact 0 and 1 before a log.

Call it as np.clip(x, lo, hi)

import numpy as np
p = np.array([0.0, 0.3, 1.0])
eps = 1e-12
print(np.clip(p, eps, 1 - eps))
print(-np.log(np.clip(p, eps, 1 - eps)))
output[1.e-12 3.e-01 1.e+00]
[2.76310211e+01 1.20397280e+00 9.99977878e-13]
Watch out
  • Clipping gradients is the same idea applied to gradient values; clipping by norm is different and uses np.linalg.norm.

Official documentation ↗

§ np.sin, np.cos, np.arccos, np.degrees

Trigonometry on each element, in radians. In these notes they appear only to turn a cosine similarity into an angle and to draw curves.

Call it as np.cos(x) np.sin(x) np.arccos(c) np.degrees(radians)

import numpy as np
cos_sim = 0.5
print(np.degrees(np.arccos(cos_sim)))   # the angle whose cosine is 0.5
print(np.cos(0), np.cos(np.pi))
output60.00000000000001
1.0 -1.0
Watch out
  • Everything is in radians. np.degrees converts for humans.
  • np.arccos of a value slightly outside [-1, 1] from rounding gives nan; clip first.

Official documentation ↗

§ np.e, np.pi, np.inf, np.nan

Four built-in values: e ≈ 2.718, π ≈ 3.14159, infinity, and 'not a number', the value NumPy uses for undefined results and missing data.

Call it as np.e np.pi np.inf np.nan

import numpy as np
print(np.e, np.pi)
print(np.inf > 1e308, -np.inf < 0)
print(np.nan == np.nan)          # False! nan is not equal to anything
x = np.array([1.0, np.nan, 3.0])
print(np.isnan(x), np.nanmean(x))
output2.718281828459045 3.141592653589793
True True
False
[False  True False] 2.0
Watch out
  • nan is not equal to itself, so use np.isnan to find it. Any arithmetic with nan gives nan, and x.mean() with one nan is nan; np.nanmean ignores them.
  • np.inf - np.inf is nan.

Official documentation ↗

Reductions: one number from many

A reduction collapses an array to fewer numbers: a sum, a mean, the position of the largest. The axis argument says which direction to collapse, and it is the single most asked-about NumPy idea in interviews.

§ np.sum and .sum()

Adds up the elements. With no axis, all of them, giving one number. axis=0 adds down the columns (collapsing rows); axis=1 adds across each row. Also written as a.sum().

Call it as np.sum(a, axis=None, keepdims=False) a.sum(axis=...)

import numpy as np
M = np.array([[1, 2, 3],
              [4, 5, 6]])
print(M.sum())
print(M.sum(axis=0))      # column totals, shape (3,)
print(M.sum(axis=1))      # row totals, shape (2,)
print(M.sum(axis=1, keepdims=True).shape)
output21
[5 7 9]
[ 6 15]
(2, 1)
Watch out
  • axis is the axis that disappears. axis=0 on a (2, 3) array gives shape (3,): the rows were collapsed.
  • keepdims=True keeps the collapsed axis with length 1, so the result still broadcasts against the original. Softmax needs this.

Official documentation ↗

§ np.mean and .mean()

The average. Same axis rules as sum. The mean of a boolean array is the fraction that is True.

Call it as np.mean(a, axis=None) a.mean(axis=...)

import numpy as np
X = np.array([[1.0, 100.0], [2.0, 300.0], [3.0, 500.0]])
print(X.mean())
print(X.mean(axis=0))     # per-feature means, the thing a scaler learns
errors = np.array([-20.0, -30.0, -20.0])
print((errors ** 2).mean())   # mean squared error
print((np.array([1, 0, 1]) == np.array([1, 1, 1])).mean())   # accuracy
output151.0
[  2. 300.]
566.6666666666666
0.6666666666666666
Watch out
  • Mean over axis=0 of an (n, d) matrix gives d numbers: one per column, averaged over the rows. Get this backwards and your scaler standardises examples instead of features.

Official documentation ↗

§ np.median and np.percentile

The median is the middle value after sorting; it ignores outliers that drag the mean. np.percentile gives any cut point, such as the 25th and 75th percentiles used by the robust scaler.

Call it as np.median(a) np.percentile(a, q)

import numpy as np
income = np.array([30, 35, 40, 45, 50, 1000])
print(income.mean(), np.median(income))
print(np.percentile(income, [25, 50, 75]))
output200.0 42.5
[36.25 42.5  48.75]
Watch out
  • With an even count the median is the average of the two middle values.

Official documentation ↗

§ np.std, np.var and .std()

Spread. Variance is the mean squared distance from the mean; standard deviation is its square root, in the original units. Both take axis.

Call it as a.std(axis=None, ddof=0) a.var(...)

import numpy as np
x = np.array([2.0, 4.0, 4.0, 4.0, 5.0, 5.0, 7.0, 9.0])
print(x.var(), x.std())
print(x.std(ddof=1))     # the 'sample' version divides by n-1
X = np.array([[1.0, 100.0], [2.0, 300.0], [3.0, 500.0]])
print((X - X.mean(axis=0)) / X.std(axis=0))   # standardisation
output4.0 2.0
2.138089935299395
[[-1.22474487 -1.22474487]
 [ 0.          0.        ]
 [ 1.22474487  1.22474487]]
Watch out
  • NumPy divides by n by default (ddof=0); pandas divides by n−1 by default. The two disagree on small data.
  • A constant column has std 0, and dividing by it gives nan. Scalers guard against this.

Official documentation ↗

§ np.max, np.min, .max(), .min()

The largest or smallest element, overall or along an axis.

Call it as a.max(axis=None) a.min(axis=None)

import numpy as np
z = np.array([[2.0, 1.0, 0.0], [5.0, 1.0, -3.0]])
print(z.max(), z.min())
print(z.max(axis=1))
print(z.max(axis=1, keepdims=True))
x = np.array([1.0, 50.0, 100.0])
print((x - x.min()) / (x.max() - x.min()))   # min-max scaling to [0, 1]
output5.0 -3.0
[2. 5.]
[[2.]
 [5.]]
[0.         0.49494949 1.        ]
Watch out
  • np.max reduces one array; np.maximum compares two. They are different functions.

Official documentation ↗

§ np.argmax, np.argmin, .argmax()

The position of the largest (or smallest) element, not its value. On a matrix of class scores, argmax(axis=1) gives each row's predicted class.

Call it as a.argmax(axis=None) a.argmin(axis=None)

import numpy as np
scores = np.array([[0.1, 0.7, 0.2],
                   [0.8, 0.1, 0.1]])
print(scores.argmax(axis=1))     # predicted class per row
print(scores.argmax())           # flat position when no axis is given
losses = np.array([74.1, 22.2, 41.7])
print(losses.argmin(), losses.min())
output[1 0]
3
1 22.2
Watch out
  • With no axis it returns one flat position, even on a matrix; use np.unravel_index or give an axis.
  • Ties go to the first occurrence.

Official documentation ↗

§ np.sort and np.argsort

np.sort returns the elements in increasing order. np.argsort returns the positions that would sort them, which is what you need to reorder a second array the same way (for example, to rank neighbours by distance).

Call it as np.sort(a) np.argsort(a) np.argsort(a)[::-1]

import numpy as np
dist = np.array([2.5, 0.4, 1.7])
labels = np.array(['cat', 'dog', 'cat'])
print(np.sort(dist))
order = np.argsort(dist)
print(order)
print(labels[order])          # labels of the nearest first
print(np.argsort(dist)[::-1][:2])   # positions of the two largest
output[0.4 1.7 2.5]
[1 2 0]
['dog' 'cat' 'cat']
[0 2]
Watch out
  • There is no 'descending' flag; reverse with [::-1].
  • np.sort returns a copy; a.sort() sorts in place and returns None.

Official documentation ↗

§ np.prod

Multiplies all the elements together. Products of many probabilities underflow to zero, which is the reason losses use sums of logs instead.

Call it as np.prod(a, axis=None)

import numpy as np
p = np.array([0.9, 0.8, 0.95])
print(np.prod(p))
print(np.prod(np.full(1000, 0.001)))      # underflows to exactly 0
print(np.log(np.full(1000, 0.001)).sum())   # fine in log space
output0.684
0.0
-6907.755278982137
Watch out
  • Products of many numbers below 1 underflow to 0; work in logs.

Official documentation ↗

Linear algebra: dot, matmul, norm, solve

The functions behind the vector and matrix notes. Each one is a formula from those notes with the loop removed.

§ np.dot

The dot product of two vectors: multiply matching elements and add. Also written v @ w. For 2-D inputs it does matrix multiplication, but @ is clearer for that.

Call it as np.dot(a, b) a @ b

import numpy as np
w = np.array([180.0, 1.0])
x = np.array([1.5, 23.33])
print(np.dot(w, x), w @ x, (w * x).sum())   # three spellings of one thing
u = np.array([1.0, 0.0])
v = np.array([0.0, 1.0])
print(u @ v)        # perpendicular -> 0
output293.33 293.33 293.33
0.0
Watch out
  • np.dot(a, b) with two 1-D arrays is a number. With 2-D arrays it multiplies matrices; prefer @ there so the reader knows what you meant.

Official documentation ↗

§ @ and np.matmul

Matrix multiplication. The inner dimensions must match: (m, n) @ (n, p) gives (m, p). Each output entry is a dot product of a row of the left matrix with a column of the right one.

Call it as A @ B np.matmul(A, B)

import numpy as np
# 3 houses, bias column + size
X = np.array([[1.0, 1.0], [1.0, 1.5], [1.0, 2.0]])
w = np.array([23.33, 180.0])
# (3,2) @ (2,) -> (3,): all predictions at once
print(X @ w)
A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [1, 0]])
print(A @ B)
print(B @ A)                 # order matters
print(A * B)                 # NOT matrix multiplication
output[203.33 293.33 383.33]
[[2 1]
 [4 3]]
[[3 4]
 [1 2]]
[[0 2]
 [3 0]]
Watch out
  • * is element-wise. @ is matrix multiplication. Mixing them up is the most common NumPy bug in ML code.
  • Shape error 'shapes (3,2) and (3,2) not aligned' means the inner dimensions differ; transpose one side.
  • A @ B is not B @ A in general.

Official documentation ↗

§ np.linalg.norm

The length of a vector. By default the ordinary Euclidean length (square root of the sum of squares, the L2 norm). ord=1 gives the L1 norm (sum of absolute values). With axis it gives the length of every row or column.

Call it as np.linalg.norm(v) np.linalg.norm(v, ord=1) np.linalg.norm(M, axis=1)

import numpy as np
v = np.array([3.0, 4.0])
print(np.linalg.norm(v))
print(np.linalg.norm(v, ord=1), np.linalg.norm(v, ord=np.inf))
a = np.array([1.0, 2.0])
b = np.array([4.0, 6.0])
print(np.linalg.norm(a - b))              # distance between two points
# cosine similarity
print(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))
E = np.array([[3.0, 4.0], [1.0, 0.0]])
print(np.linalg.norm(E, axis=1))          # length of each row
print(E / np.linalg.norm(E, axis=1, keepdims=True))   # unit rows
output5.0
7.0 4.0
5.0
0.9922778767136677
[5. 1.]
[[0.6 0.8]
 [1.  0. ]]
Watch out
  • On a matrix with no axis it returns the Frobenius norm (all entries squared, summed, rooted), not a per-row result.
  • Distance is the norm of the difference; cosine similarity is the dot product divided by both norms.

Official documentation ↗

§ np.linalg.inv

The inverse of a square matrix: the matrix that undoes it, so that A @ inv(A) is the identity. Appears in the closed-form least-squares formula. In real code you rarely call it; solve or lstsq are more accurate.

Call it as np.linalg.inv(A)

import numpy as np
A = np.array([[2.0, 1.0], [1.0, 3.0]])
Ai = np.linalg.inv(A)
print(Ai)
print(np.allclose(A @ Ai, np.eye(2)))
try:
    # rows are multiples: no inverse
    np.linalg.inv(np.array([[1.0, 2.0], [2.0, 4.0]]))
except np.linalg.LinAlgError as err:
    print('LinAlgError:', err)
output[[ 0.6 -0.2]
 [-0.2  0.4]]
True
LinAlgError: Singular matrix
Watch out
  • A matrix whose rows (or columns) are multiples of each other has no inverse; inv raises LinAlgError. In regression this means two features carry the same information.
  • inv(XᵀX) @ Xᵀ @ y works on paper; np.linalg.lstsq is the numerically safe way to get the same answer.

Official documentation ↗

§ np.linalg.solve

Solves A x = b for x without computing an inverse. Faster and more accurate than inv(A) @ b.

Call it as np.linalg.solve(A, b)

import numpy as np
A = np.array([[2.0, 1.0], [1.0, 3.0]])
b = np.array([3.0, 5.0])
x = np.linalg.solve(A, b)
print(x)
print(np.allclose(A @ x, b))
output[0.8 1.4]
True
Watch out
  • A must be square and invertible; for tall matrices (more equations than unknowns) use lstsq.

Official documentation ↗

§ np.linalg.lstsq

Least squares: finds the weights that make X @ w as close as possible to y. This is linear regression in one call, and it works even when the exact solution does not exist.

Call it as w, residuals, rank, sv = np.linalg.lstsq(X, y, rcond=None)

import numpy as np
x = np.array([1.0, 1.5, 2.0])
y = np.array([200.0, 300.0, 380.0])
X = np.column_stack([np.ones(3), x])          # bias column, then size
w, res, rank, sv = np.linalg.lstsq(X, y, rcond=None)
print(np.round(w, 2))         # [b, w] = [23.33, 180]
print(res / 3)                # sum of squared residuals / n = MSE
output[ 23.33 180.  ]
[22.22222222]
Watch out
  • It returns four things; the weights are the first. Pass rcond=None to silence the deprecation warning.
  • Remember the column of ones, or you are fitting a line through the origin.

Official documentation ↗

§ np.polyfit and np.polyval

Fits a polynomial curve to points. A polynomial of degree d is a curve built from powers of x up to x to the d: degree 1 is a straight line (w·x + b), degree 2 a parabola (one bend), and a higher degree can bend more times. polyfit returns the best coefficients by least squares, highest power first; polyval uses them to compute the curve at new x values.

Call it as coeffs = np.polyfit(x, y, degree) y_hat = np.polyval(coeffs, x_new)

import numpy as np
sizes = np.array([1.0, 1.5, 2.0])
prices = np.array([200.0, 300.0, 380.0])

line = np.polyfit(sizes, prices, 1)     # degree 1: a straight line
# [slope, intercept] = [180, 23.33]
print(line.round(2))
print(np.polyval(line, 1.8).round(2))   # price predicted for a new house

# degree 2: three points, three coefficients
curve = np.polyfit(sizes, prices, 2)
# passes through every point exactly
print(np.polyval(curve, sizes).round(2))
output[180.    23.33]
347.33
[200. 300. 380.]
Watch out
  • The coefficients come highest power first: for degree 1 that is [slope, intercept], the opposite of the usual b, w order.
  • With as many coefficients as points the curve passes through every point: zero training error and usually a bad model. That is overfitting in one line.
  • High degrees on large x values give huge powers and a RankWarning; that is a sign the degree is too high.

Official documentation ↗

§ np.linalg.matrix_rank

How many independent rows (or columns) a matrix has. A feature matrix with rank less than its number of columns has a redundant feature, and XᵀX cannot be inverted.

Call it as np.linalg.matrix_rank(M)

import numpy as np
# column 2 = 2 x column 1
X = np.array([[1.0, 2.0], [2.0, 4.0], [3.0, 6.0]])
print(np.linalg.matrix_rank(X))
X2 = np.array([[1.0, 2.0], [2.0, 5.0], [3.0, 6.0]])
print(np.linalg.matrix_rank(X2))
output1
2
Watch out
  • Rank is computed numerically; a column that is almost a multiple of another still counts as independent but makes the fit unstable.

Official documentation ↗

§ np.linalg.LinAlgError

The error NumPy raises when a linear-algebra operation is impossible, most often inverting a singular matrix. Catch it when fitting on data that may have duplicate features.

Call it as try: ... except np.linalg.LinAlgError:

import numpy as np
S = np.array([[1.0, 2.0], [2.0, 4.0]])
try:
    np.linalg.inv(S)
except np.linalg.LinAlgError as err:
    print('cannot invert:', err)
outputcannot invert: Singular matrix

Official documentation ↗

§ np.linalg.eig, np.linalg.eigh, np.linalg.svd

Eigen-decomposition and the singular value decomposition. You will meet them in week 4 for PCA: the eigenvectors of the covariance matrix are the directions of most spread. eigh is for symmetric matrices (like covariance) and returns sorted real results.

Call it as vals, vecs = np.linalg.eigh(C) U, s, Vt = np.linalg.svd(X)

import numpy as np
X = np.array([[2.0, 0.0], [0.0, 1.0], [-2.0, 0.0], [0.0, -1.0]])
C = X.T @ X / len(X)           # covariance (data already centred)
vals, vecs = np.linalg.eigh(C)
print(vals)                    # variances along each direction, ascending
print(vecs)                    # the directions, as columns
U, s, Vt = np.linalg.svd(X, full_matrices=False)
print(s ** 2 / len(X))         # same variances from the SVD
output[0.5 2. ]
[[0. 1.]
 [1. 0.]]
[2.  0.5]
Watch out
  • eigh returns eigenvalues in ascending order; PCA wants the largest first, so reverse.
  • Use eigh, not eig, on symmetric matrices: it is faster and never returns tiny imaginary parts.

Official documentation ↗

Random numbers

Fake data, shuffled splits and starting weights all need random numbers. Modern NumPy puts them behind a generator object made once with a seed, so results repeat exactly.

§ np.random.default_rng

Makes a random-number generator. Give it a seed (any integer) and every run produces the same 'random' numbers, which is what makes experiments repeatable. All the rng.* calls below come from this object.

Call it as rng = np.random.default_rng(seed)

import numpy as np
rng = np.random.default_rng(0)
print(rng.random(3))
rng2 = np.random.default_rng(0)
print(rng2.random(3))      # identical: same seed, same numbers
print(np.random.default_rng(1).random(3))   # different seed
output[0.63696169 0.26978671 0.04097352]
[0.63696169 0.26978671 0.04097352]
[0.51182162 0.9504637  0.14415961]
Watch out
  • The old style, np.random.seed(0) and np.random.randn(...), still works but is discouraged; new code uses a Generator.
  • Make one generator and pass it around; making a new one with the same seed inside a loop gives the same numbers every iteration.

Official documentation ↗

§ rng.normal

Random numbers from a bell curve with the given mean and standard deviation. Used for noise in fake data and for starting weights.

Call it as rng.normal(loc=0.0, scale=1.0, size=None)

import numpy as np
rng = np.random.default_rng(0)
noise = rng.normal(0, 1, size=5)
print(np.round(noise, 3))
W = rng.normal(0, 0.01, size=(2, 3))    # small starting weights
print(W.shape)
big = rng.normal(10, 2, size=100_000)
print(round(big.mean(), 2), round(big.std(), 2))
output[ 0.126 -0.132  0.64   0.105 -0.536]
(2, 3)
10.0 2.0
Watch out
  • The second argument is the standard deviation, not the variance.
  • rng.standard_normal(size) is the shortcut for mean 0, std 1.

Official documentation ↗

§ rng.uniform

Random numbers spread evenly between low and high. rng.random() is the same on [0, 1).

Call it as rng.uniform(low=0.0, high=1.0, size=None)

import numpy as np
rng = np.random.default_rng(0)
x = rng.uniform(0, 10, size=5)
print(np.round(x, 2))
print(rng.uniform(-1, 1, size=(2, 2)).shape)
output[6.37 2.7  0.41 0.17 8.13]
(2, 2)
Watch out
  • high is excluded, like a range.

Official documentation ↗

§ rng.integers

Random whole numbers from low up to (not including) high. For random labels or random indices.

Call it as rng.integers(low, high=None, size=None)

import numpy as np
rng = np.random.default_rng(0)
print(rng.integers(0, 2, size=10))     # ten coin flips as 0/1
print(rng.integers(0, 100, size=3))
output[1 1 1 0 0 0 0 0 0 1]
[64 91 50]
Watch out
  • high is excluded; rng.integers(0, 2) gives 0 or 1, never 2.

Official documentation ↗

§ rng.permutation and rng.shuffle

permutation returns the numbers 0..n−1 in random order (or a shuffled copy of an array). shuffle reorders an array in place. Either one is the heart of a train/test split.

Call it as rng.permutation(n) rng.shuffle(a)

import numpy as np
rng = np.random.default_rng(0)
idx = rng.permutation(10)
print(idx)
train, test = idx[:8], idx[8:]
print(train, test)               # 80/20 split of row positions
a = np.arange(5)
rng.shuffle(a)
print(a)
output[4 6 2 7 3 5 9 0 8 1]
[4 6 2 7 3 5 9 0] [8 1]
[0 2 3 4 1]
Watch out
  • Shuffle the index, then apply the same index to X and y; shuffling X and y separately breaks their pairing.

Official documentation ↗

§ rng.choice

Picks elements at random from an array, with or without replacement. Bootstrap samples and random mini-batches both use it.

Call it as rng.choice(a, size=None, replace=True, p=None)

import numpy as np
rng = np.random.default_rng(0)
# 3 distinct positions out of 5
print(rng.choice(5, size=3, replace=False))
print(rng.choice(['a', 'b', 'c'], size=5))     # with replacement
print(rng.choice([0, 1], size=10, p=[0.9, 0.1]))   # weighted
output[3 4 2]
['a' 'a' 'a' 'a' 'c']
[1 0 0 0 1 0 0 0 0 0]
Watch out
  • replace=True by default, so repeats are possible unless you say otherwise.

Official documentation ↗

Checking and printing

Floating-point numbers are rarely exactly equal. These are the honest ways to compare them, and a way to keep printed output short.

§ np.errstate

Temporarily changes what NumPy does about overflow, division by zero and invalid results (like 0/0). By default it prints a RuntimeWarning and carries on with inf or nan; inside with np.errstate(all="ignore"): the warnings are hidden, used when a demo blows up on purpose.

Call it as with np.errstate(all="ignore"): ...

import numpy as np
with np.errstate(all="ignore"):
    print(np.array([1.0, 0.0]) / 0.0)      # inf and nan, no warning
    print(np.exp(np.array([1000.0])))      # overflow to inf
output[inf nan]
[inf]
Watch out
  • Hiding warnings hides real bugs too; use it only where the inf or nan is expected and explained.

Official documentation ↗

§ np.isclose and np.allclose

Floating-point arithmetic has rounding error, so 0.1 + 0.2 is not 0.3. isclose compares element by element within a small tolerance and returns a boolean array; allclose returns a single True if every element is close.

Call it as np.isclose(a, b) np.allclose(a, b, rtol=1e-5, atol=1e-8)

import numpy as np
print(0.1 + 0.2 == 0.3)
print(np.isclose(0.1 + 0.2, 0.3))
A = np.array([[2.0, 1.0], [1.0, 3.0]])
print(np.allclose(A @ np.linalg.inv(A), np.eye(2)))
total = np.log(np.array([0.9, 0.8, 0.95])).sum()
print(np.isclose(total, np.log(0.684), atol=1e-3))
outputFalse
True
True
True
Watch out
  • Never test floats with ==. Every 'check it numerically' line in these notes uses isclose or allclose.
  • The default tolerance is relative, so comparing against 0 needs an explicit atol.

Official documentation ↗

§ np.set_printoptions

Controls how arrays print: number of decimals, whether to use scientific notation, how many elements before NumPy abbreviates with '...'.

Call it as np.set_printoptions(precision=3, suppress=True)

import numpy as np
x = np.array([0.66524096, 0.24472847, 0.09003057, 1e-10])
print(x)
np.set_printoptions(precision=3, suppress=True)
print(x)
output[6.6524096e-01 2.4472847e-01 9.0030570e-02 1.0000000e-10]
[0.665 0.245 0.09  0.   ]
Watch out
  • It is global for the session; call it once at the top of a notebook.

Official documentation ↗

§ .fill and in-place updates

a.fill(value) sets every element to one value without making a new array. a += b, a *= 2 and a[mask] = 0 also change the array in place. Useful in training loops where a gradient array is reused.

Call it as a.fill(0) a += b a[mask] = value

import numpy as np
g = np.ones(3)
g.fill(0)
print(g)
w = np.array([1.0, 2.0])
# one gradient-descent step, in place
w -= 0.1 * np.array([10.0, 20.0])
print(w)
x = np.array([-1.0, 2.0, -3.0])
x[x < 0] = 0
print(x)
output[0. 0. 0.]
[0. 0.]
[0. 2. 0.]
Watch out
  • In-place on an integer array with a float value truncates: np.array([1, 2]) -= 0.5 raises an error in modern NumPy, which is a feature.

Official documentation ↗

Practice

The fastest way to make this stick is to type each example yourself and change one thing. Then do the NumPy problems on Deep-ML (they are small functions, each one a NumPy exercise) and the official beginner guide, which is short and covers the same ideas in the same order.