AI Glossary

AI terms, without the unnecessary jargon.

Algorithm: A formal sequence of instructions to solve a problem. Predates AI and remains ubiquitous without it.; An algorithm is a finite, unambiguous sequence of instructions to solve a problem. Most systems you use daily (sorting, search, simple recommendations) rely on deterministic algorithms, not machine learning.; mechanics
Machine learning: The computer learns from examples, without hardcoded rules.; Machine learning is an AI subfield where the system infers rules from data rather than following explicit instructions. It includes regression, classification, clustering, etc.; mechanics
Deep learning: Machine learning with deep (multi-layer) neural networks.; Deep learning uses artificial neural networks with multiple hidden layers. It enabled recent breakthroughs in image recognition, speech, and machine translation.; mechanics
Neural network: A computational architecture very loosely inspired by the biological brain.; An artificial neural network is a graph of nodes (neurons) organized in layers, transforming inputs into outputs via parameters (weights) adjusted during training. The brain analogy is pedagogically useful but misleading: there is no consciousness, no understanding.; mechanics
Model: A trained program that produces outputs from inputs. Not an intelligence.; In the AI context, a model is the result of training on a dataset. It encodes statistical correlations. GPT-4, Mistral, Llama are models. They produce plausible outputs, not reasoned conclusions.; mechanics
Training: The phase where the model learns from data. Slow, expensive, done once (or rarely).; Training adjusts a neural network's parameters to minimize error on a dataset. For large language models, this can mobilize thousands of GPUs for weeks and cost tens of millions of dollars.; mechanicseconomics
Dataset: The collection of examples on which the model trains.; A dataset is the raw material for learning. Its quality, size, and representativeness directly determine model quality. AI only learns what is in the data.; mechanics
Parameters: The internal numerical values adjusted during training. Often expressed in billions.; Parameters (or weights) are a model's internal variables. A 70-billion-parameter LLM stores 70 billion floating-point numbers. More parameters means more capacity to model complex patterns — but also more memory and compute.; mechanics
Overfitting: The model learned the training data too well and generalizes poorly.; Overfitting occurs when a model memorizes training examples instead of extracting general rules. It performs well on seen data but fails on new data. A sign of a too-small dataset or an overly complex model.; mechanicsrisks
Inference: Using the trained model to produce a response. Continuous cost.; Inference is the operational phase: submit an input, get an output. Unlike training (one-time cost), inference happens on every request and generates ongoing variable costs.; mechanicseconomics
Token: The text unit a LLM processes. Roughly 3/4 of an English word.; A token is a model's processing unit. Text is split into tokens before processing. API prices are charged per token (input + output). 1,000 tokens ≈ 750 English words.; mechanicseconomics
LLM: Large Language Model: GPT-4, Claude, Llama, Mistral... Predicts the most probable next token.; A Large Language Model is a neural network trained on vast text corpora to predict the next token. GPT-4, Claude, Llama, Mistral are examples. They are statistical completion systems, not intelligences that understand.; mechanics
Embedding: A numerical representation of text in a vector space.; An embedding is the conversion of text (word, sentence, document) into a numerical vector. Semantically similar texts have close vectors. Embeddings are central to semantic search and RAG systems.; mechanics
Vectorization: The process of converting data into numerical vectors.; Vectorization transforms unstructured data (text, images, audio) into mathematically manipulable numerical representations. Fundamental for semantic search, similarity comparison, and recommendation systems.; mechanics
RAG: Retrieval-Augmented Generation: retrieve documents, give them to the LLM to answer.; RAG combines a vector search engine with an LLM. Instead of stuffing everything into the context, relevant passages are retrieved on the fly. Reduces hallucinations on specialized corpora, but doesn't eliminate them.; mechanicsusage
Fine-tuning: Additional training of an existing model on specific data.; Fine-tuning takes a pre-trained model and continues training on a more targeted dataset (domain, style, format). Less expensive than full training, but more expensive than prompting. Only worthwhile if format or vocabulary cannot be captured by a prompt.; mechanicseconomics
Hallucination: The model asserts something false with confidence. A design property, not a bug to fix.; A hallucination is a factually incorrect output produced confidently by a model. It is not a bug: it is a direct consequence of prediction-based operation. The model generates what is statistically probable, not what is true. It will not disappear with the next version.; mechanicsrisks
Agent: An LLM that can use tools and chain actions.; An AI agent is a system that uses an LLM to decide what actions to take (API calls, web search, file writing) and executes them in sequence. Useful for automating complex tasks, but also amplifies error risk: hallucinations propagate through the action chain.; mechanicsrisks
Reasoning: What recent models call 'thinking': staged completion, not actual thought.; Reasoning models (o1, o3, R1...) generate intermediate token chains before the final answer. This is a more developed completion technique, not reasoning in the human sense. The model has no internal world representation: it generates plausible text at each step.; mechanicsreality
GPU: Massively parallel graphics processor. The basic hardware for training and running AI models.; A GPU (Graphics Processing Unit) executes thousands of operations in parallel, making it suitable for the intensive matrix calculations of machine learning. NVIDIA H100/A100 GPUs dominate the AI market. Chronic shortage since 2022.; mechanicssovereignty
APU: A processor integrating CPU and GPU on the same chip. Enables running models locally.; An APU (Accelerated Processing Unit) combines CPU and GPU on the same die. Apple Silicon (M-series) and AMD Ryzen AI are examples. They enable running 7 to 30-billion-parameter models locally with reasonable power consumption.; mechanicssovereignty
On-premise: Infrastructure runs on your own hardware, under your control.; On-premise (on-prem) means deploying in the organization's own infrastructure, as opposed to the cloud. For AI, this involves owning or renting GPU/APU hardware. The only option for guaranteeing data never leaves your perimeter.; sovereignty
Open weights: The model weights are public, but not necessarily the training code or data.; An open weights model (Llama, Mistral, DeepSeek...) publishes its weights, allowing download and local deployment. This is NOT the same as open source: training code and data may remain proprietary. The distinction matters: some 'open' models prohibit commercial use.; sovereignty
Open source: Code, data, and weights are all public and freely reusable.; Strictly speaking, a truly open source AI model publishes training code, training data, AND weights. Very few models reach this level. Most so-called 'open' models are open weights only.; sovereignty
Sovereignty: The ability to control your AI tools without critical dependency on a third party.; Digital sovereignty over AI means an organization's (or state's) ability to control its AI systems: access to data, models, and infrastructure. It doesn't mean building everything yourself, but avoiding unacceptable dependencies on third-party actors.; sovereignty
Bias: Systematic error in a model's outputs, inherited from data or design.; Bias in an AI model is a systematic tendency to produce incorrect or discriminatory results for certain groups or contexts. It can come from training data, architecture, or the alignment process. It cannot be fully eliminated, only reduced and documented.; risks