Tensor

A mathematical object analogous to but more general than a vector, widely used in machine learning as multi-dimensional data arrays.

How it works

A tensor is a multi-dimensional array that generalises scalars (0D), vectors (1D), and matrices (2D) to arbitrary dimensions. In deep learning, data flows through networks as tensors: a batch of images might be a 4D tensor (batch × height × width × channels), and the weight matrix of a Transformer attention head is a 3D tensor. All major deep learning frameworks (PyTorch, TensorFlow) are built around efficient tensor operations, which are parallelised on GPU hardware.

Why it matters

Tensors are the fundamental data structure of deep learning. Every computation in a neural network — forward passes, backpropagation, weight updates — is expressed as tensor operations. Understanding tensors is necessary for debugging shape mismatches (the most common source of deep learning bugs), for reasoning about memory requirements (tensor size determines GPU memory usage), and for implementing custom model architectures efficiently.