A processing unit in an NVIDIA GPU used for AI and high-performance computing (HPC). There are several hundred Tensor cores in a GPU, which compute values in parallel.
Not Just NVIDIA
NVIDIA added Tensor cores to its GPUs after Google introduced the Tensor Processing Unit, designed from scratch for AI. Other companies also make math acceleration hardware for AI, including AMD and Amazon.
Multiply and Add
A new array is created from two arrays by multiplying and adding 16- or 32-bit floating point values in parallel. Optimized for mixed precision, Tensor cores work with both floating point and integer math.
NVIDIA CUDA and Tensor Cores
NVIDIA's CUDA Cores were originally designed for graphics processing, and although they are used in AI applications, Tensor cores were designed specifically for AI. Tensor also supports more mathematical data types than CUDA (see
TF32). Both CUDA and Tensor cores are found in NVIDIA GPUs. See
CUDA core,
matrix multiplication,
GPU,
Tensor Processing Unit and
Tensor chip.