AI glossary

Accuracy is the fraction of all predictions the model got right, which becomes misleading when one class is rare.

An activation function is the small bending step applied to each neuron's output, and it is what lets a deep network learn curved patterns instead of…

Adam is the default optimizer for deep learning — momentum plus an automatic per-weight step size.

AGI is a hypothetical AI that could learn any intellectual task a person can learn, instead of being good at one narrow job.

An AI agent is a language model that can pick and run tools in a loop until a goal is finished, instead of replying once and stopping.

Alignment is the work of making AI systems actually pursue what their builders and users intend — helpful, honest, and safe behaviour.

Attention is the step where a model decides which other words in the input matter most for the word it is handling right now.

An autoencoder is a network trained to squeeze data into a small code and rebuild the original from that code alone.

Backpropagation works out how much each weight in a network contributed to the error, by passing the error backwards through the layers.

A baseline is the simplest sensible solution to your problem, built first, so every fancier model has something honest to beat.

Batch normalization rescales each layer's outputs using the current batch's statistics, keeping the numbers in a healthy range so training stays…

Batch size is how many training examples the model looks at before it updates its weights once.

Batching groups multiple inputs and processes them together in one pass, the key to keeping GPUs busy in both training and serving.

Beam search keeps the several best partial outputs alive at each step, so one early wrong word cannot ruin the whole answer.

A benchmark is a shared, fixed test that lets different models be compared on the same questions under the same rules.

BERT is a language model that reads a whole sentence in both directions at once to understand it, rather than writing new text one word at a time.

The bias-variance tradeoff is the balance between a model too rigid to learn the pattern and one flexible enough to memorise noise.

BPE builds a tokenizer's vocabulary by repeatedly merging the most frequent pair of adjacent pieces, so common words become single tokens.

Chain-of-thought prompting asks the model to write out its reasoning steps before the answer, sharply improving accuracy on multi-step problems.

A checkpoint is a saved snapshot of a model's weights at a moment in training — the file you resume from, evaluate, or ship.

Chunking is splitting documents into retrieval-sized pieces for RAG — the unglamorous decision that quietly determines answer quality.

Class imbalance is when one outcome vastly outnumbers the other in your data, letting lazy models score high while learning nothing.

Classification is predicting which category something belongs to — spam or not, cat or dog, approve or reject.

Clustering groups similar examples together without being told what the groups are — the algorithm discovers them.

A CNN learns small visual patterns and slides them across an image, so it can find the same pattern anywhere in the picture.

A confusion matrix is a small table showing how many predictions were right and exactly which kinds of mistakes the model made.

The context window is the maximum amount of text, counted in tokens, that a model can hold in view at one time.

Cross-validation tests a model on several different train/test splits and averages the scores, so one lucky split cannot fool you.

Data augmentation creates extra training examples by making label-preserving changes to the ones you have — flips, crops, noise, rewording.

Data drift is when the real-world data a deployed model receives slowly stops resembling the data it was trained on.

Data labelling is the human work of writing the correct answer onto each training example — slow, costly, and quality-critical.

Data leakage is when information that will not exist at prediction time sneaks into training, producing scores that collapse in production.

A decision tree predicts by asking a sequence of yes/no questions about the features, like a flowchart learned from data.

A diffusion model generates images by learning to remove noise step by step, starting from pure static and sculpting it into a picture.

Distillation trains a small student model to imitate a large teacher model, keeping much of the capability at a fraction of the cost.

DPO teaches a model human preferences directly from chosen-versus-rejected answer pairs, skipping RLHF's reward model and RL loop entirely.

Dropout randomly switches off a fraction of neurons during each training step, so no single neuron can be relied on too much.

Early stopping ends training when performance on held-out data stops improving, keeping the model from sliding into memorisation.

An embedding is a list of numbers representing a piece of text, an image or an item, arranged so that similar things land close together.

An ensemble combines several models' predictions into one, usually beating any single member.

An epoch is one complete pass of the entire training dataset through the model.

Evals are systematic tests for LLM behaviour — your own graded question set, run on every change, because "it seems better" is not measurement.

The F1 score folds precision and recall into one number, and it stays low unless both are good.

A feature is one measurable property of the thing you are predicting about — one column in your data table.

Feature engineering is creating better input columns from raw data, using what you know about the problem.

Feature scaling puts all input columns on comparable ranges, so no feature dominates purely because its numbers are bigger.

Few-shot prompting means including a handful of worked examples in the prompt, so the model copies the pattern instead of guessing your intent.

Fine-tuning is taking a model that already knows a lot and training it a little more on your own examples so it fits your task.

FlashAttention computes exact attention in a memory-smart order, avoiding the huge intermediate score matrix and making long contexts affordable.

FLOPS measure how many arithmetic operations per second hardware can perform — the standard currency for compute in AI.

A foundation model is one huge pretrained model that many different applications adapt and build on, instead of everyone training from scratch.

Function calling lets an LLM request that your code run a specific function with specific arguments, connecting language to real actions and live…

A GAN trains two networks against each other, one making fakes and one spotting fakes, until the fakes look real.

GGUF is the file format for running quantized LLMs on ordinary computers — one self-contained file that llama.cpp, Ollama and LM Studio all read.

A GPU does thousands of simple calculations at once, which is exactly the shape of work neural networks need — hence its central role in AI.

Gradient boosting builds trees one after another, each trained to correct the combined errors of all the trees before it.

Gradient clipping caps the size of each update during training, so one extreme batch cannot blow the model up.

Gradient descent improves a model by repeatedly taking a small step in whichever direction lowers the error.

Greedy decoding always picks the single most probable next token — deterministic, fast, and prone to repetitive, short-sighted text.

Ground truth is the answer accepted as correct, against which every model prediction is scored.

Grounding is making a model base its answers on supplied trusted sources rather than on its trained memory, so claims can be checked.

Grouped-query attention lets several query heads share one set of keys and values, shrinking the KV cache with little quality loss.

Guardrails are the checks wrapped around a model that block bad inputs and outputs — policy enforced outside the model itself.

A hallucination is when a language model states something false with exactly the same confidence it uses for something true.

A hyperparameter is a setting you choose before training starts, unlike a parameter, which the model learns during training.

Inference is the moment you use a trained model to get an answer, as opposed to training, where the model is still learning.

Instruction tuning fine-tunes a raw text-predicting model on instruction-and-answer examples, teaching it to behave like an assistant.

A jailbreak is a prompt crafted to talk a model out of its safety training, getting it to produce what it was trained to refuse.

K-means clusters data by placing k centre points and repeatedly moving each centre to the middle of the points claimed by it.

KNN classifies a new example by finding the k most similar known examples and letting them vote.

The KV cache stores each token's attention keys and values so the model never recomputes them, making generation fast but memory-hungry.

A label is the correct answer attached to a training example — the thing you want the model to learn to predict.

Latency is how long one request takes from send to answer — the waiting a single user actually feels.

Latent space is the model's internal map of compressed representations, where distance means similarity and directions can mean concepts.

Layer normalization rescales each example's features using that example alone, which is why transformers prefer it to batch norm.

The learning rate is the size of each training step — the single hyperparameter most likely to make or break a run.

Linear regression predicts a number by drawing the best straight-line relationship between the inputs and the output.

A large language model is a program trained on enormous amounts of text to predict what comes next, which turns out to be enough to answer, write and…

LLM-as-judge uses a strong language model to grade another model's outputs against a rubric, making large-scale evaluation affordable.

Logistic regression predicts the probability of a yes/no outcome by squashing a weighted sum of the inputs into the 0-1 range.

Logits are the raw, unnormalised scores a model produces for each possible output, before they are converted into probabilities.

LoRA fine-tunes a large model by training a small pair of extra matrices while the original weights stay frozen.

A loss function scores how wrong a prediction is, giving training one number to push down.

An LSTM is an RNN with learned gates that decide what to remember, what to forget, and what to output — letting memory survive long sequences.

MCP is an open standard that lets any AI assistant connect to any tool or data source through one common protocol, instead of custom integrations for…

A mixture-of-experts model holds many specialist sub-networks but activates only a few per token, buying huge capacity at a fraction of the compute.

MLOps is the discipline of running machine learning in production — versioning, deploying, monitoring and retraining models as a routine, not a…

Model serving is running a trained model as a live service that applications call — the engineering between a saved file and real users.

Momentum makes each training step a running blend of past steps, so updates build speed in consistent directions and cancel out zigzag.

Multi-head attention runs several attention operations in parallel, letting each head watch for a different kind of relationship between tokens.

A multimodal model handles more than one kind of input or output — text, images, audio, video — inside a single model.

Naive Bayes classifies by multiplying together how strongly each feature hints at each class, assuming the features do not influence each other.

NER finds and labels the names in text — people, places, organisations, dates, amounts — turning prose into structured data.

One-hot encoding turns a category into a row of zeros with a single one, so models cannot read fake order into category numbers.

ONNX is a common file format for trained models, letting a network trained in one framework run anywhere — servers, browsers, phones.

An open-weights model is one whose trained parameters are published for download, so anyone can run and fine-tune it — which is not the same as open…

The optimizer is the rule that decides how to change each weight after the gradients are computed.

An outlier is a data point that sits far from the pattern the rest of the data follows — sometimes an error, sometimes the discovery.

Overfitting is when a model memorises its training examples instead of learning the pattern, so it scores well in practice runs and badly on new data.

A parameter is one number inside a model that training adjusts — the model's whole learned knowledge is nothing but billions of these numbers.

PCA compresses many correlated features into a few new axes that keep most of the variation in the data.

PEFT is the family of methods that adapt a large model by training only a small fraction of its weights, with LoRA as the star member.

Perplexity measures how surprised a language model is by text — lower means the model predicts that text better.

Positional encoding injects word-order information into a transformer, which otherwise sees its input as an unordered set.

Precision is the fraction of your model's positive calls that were actually correct.

Pretraining is the massive first stage of building an LLM — predicting the next token across trillions of words until general capability emerges.

A prompt is everything you send to a language model — the instruction, the context, the examples — and its quality largely decides the answer's…

Prompt caching stores the computed state of a prompt's repeated prefix, so identical opening tokens are billed and processed at a fraction of the…

Prompt injection is an attack where instructions hidden in data the model reads — an email, a web page — hijack the model's behaviour.

Quantization stores a model's numbers with fewer bits, so it needs less memory and runs faster, at the cost of a small drop in quality.

RAG finds the right documents first and puts them in the prompt, so the model answers from your data instead of from memory.

A random forest trains many decision trees on different random slices of the data and lets them vote, trading explainability for reliability.

A reasoning model is an LLM trained to think through a long internal chain of steps before answering, trading time and tokens for accuracy.

Recall is the fraction of the real positive cases that your model managed to find.

Red-teaming is deliberately attacking your own AI system to find the failures — jailbreaks, leaks, harmful outputs — before real users do.

Regression is predicting a number on a continuous scale — a price, a temperature, a delivery time.

Regularization is any extra pressure added during training that discourages a model from becoming complicated enough to memorise noise.

Reinforcement learning trains an agent by trial and error — it acts, receives rewards or penalties, and gradually prefers what worked.

Reranking is a second, more careful scoring pass over retrieved search results, promoting the truly relevant chunks before they reach the model.

A residual connection adds a layer's input directly to its output, giving both information and gradients a shortcut past the layer.

A reward model is a network trained on human preference data to score how good an answer is, standing in for human judgement at scale.

RLHF tunes a language model using human preferences between answers, which is how raw text predictors became helpful assistants.

An RNN reads a sequence one step at a time, carrying a memory of what it has seen so far in a hidden state.

ROC-AUC measures how well a model ranks positive cases above negative ones, across every possible decision threshold at once.

RoPE tells a transformer where each token sits in the sequence by rotating its query and key vectors by position-dependent angles.

Scaling laws are the observed formulas relating model size, data and compute to performance, letting labs forecast a model before spending millions…

Self-supervised learning manufactures training answers from the data itself, like hiding a word and asking the model to guess it.

Semi-supervised learning trains on a small labelled set plus a large unlabelled one, letting the cheap data amplify the expensive labels.

SGD updates the model after each small random batch of data instead of after the whole dataset, trading noisy steps for speed.

Softmax converts a list of raw scores into probabilities that are positive and sum to one, exaggerating the gaps between scores.

Speculative decoding has a small fast model draft several tokens ahead and the big model verify them in one pass — same output, delivered faster.

Streaming sends a model's answer token by token as it is generated, so the user starts reading immediately instead of staring at a spinner.

Structured output makes a model reply in a guaranteed machine-readable format — usually JSON matching your schema — instead of free prose.

Supervised learning trains a model on examples that come with correct answers attached, so it can answer for new examples later.

An SVM separates two classes with the boundary that leaves the widest possible safety margin on both sides.

Synthetic data is artificially generated training data — made by simulation or by another model — used when real data is scarce, private, or…

The system prompt is the standing instruction a model receives before any user message, defining its role, rules and boundaries.

Temperature is the dial that controls how adventurous a language model's word choices are — low is predictable, high is varied and risky.

A tensor is a grid of numbers with any number of dimensions, and it is the container every deep learning library moves around.

Throughput is how much work a system completes per unit time — requests per second, tokens per second — and it decides your cost per query.

A token is a chunk of text, often a word or part of a word, and it is the unit a language model actually reads and writes.

A tokenizer is the component that chops text into the token pieces a language model reads, using a fixed vocabulary learned from data.

Top-k sampling restricts each next-token choice to the k highest-probability candidates, a fixed-size shortlist.

Top-p sampling keeps only the smallest set of next-token candidates whose probabilities add up to p, then samples within that set.

A train/test split holds back part of your data so the model is scored on examples it has never seen.

Training data is the collection of examples a model learns from — its quality and coverage set the ceiling on everything the model can do.

Transfer learning reuses a model trained on one large task as the starting point for a related task with far less data.

The transformer is the network design behind almost every modern language model — it reads all tokens at once and uses attention to decide which ones…

Underfitting is when a model is too simple to capture the real pattern, so it performs badly even on its own training data.

Unsupervised learning finds structure in data that has no answers attached — groups, patterns, and oddities.

The validation set is the data you tune your model against during development, kept separate from both training data and the final test set.

The vanishing gradient problem is when the training signal fades to nothing as it travels back through many layers, so early layers stop learning.

A vector database stores embeddings and finds the nearest ones to a query fast, even across millions of items.

VRAM is the GPU's own on-board memory — the hard limit deciding which models you can run and train at all.

Weight decay shrinks every weight slightly at each training step, so no weight grows large without earning it.

Word2Vec was the 2013 breakthrough that learned word embeddings from raw text, famously making king − man + woman land near queen.

Zero-shot means asking a model to do a task from the instruction alone, without giving it any worked examples.