In AI engineering, 'inference' means running a trained model on new input without updating its parameters
In the vocabulary of AI engineering, inference designates the stage of a model's life that follows training: the moment when a trained model takes a live input and produces output. The model's weights are frozen; no learning occurs. The sole purpose is production: generating a useful result for a user or downstream system.
One formulation captures the distinction cleanly: training is "the one-time, capital-intensive process of teaching a model to recognize patterns across vast datasets," while inference is "the recurring, operational process of actually running that model to answer a question, transcribe a call, or drive a conversation" (Telnyx, 2026).
The two phases of a model's life
| Phase | What happens | Parameters change? | Frequency |
|---|---|---|---|
| Training | Model adjusts weights from labelled data | Yes | Occasional; per model version |
| Inference | Model processes input, returns output | No | Continuous; every user request |
This is, as noted in the Wikipedia entry on statistical inference, an inversion of the statistical sense: a statistician doing Bayesian inference is computing over parameters; an AI engineer doing inference is computing with frozen parameters. See claim-statistical-inference-meaning for this divergence.
What inference looks like in practice
Every user interaction with ChatGPT, Claude, or Gemini is an inference event. When a user submits a prompt, the system runs the model's forward pass over the tokenized input, producing output one token at a time. The model is not updated; the weights it used yesterday are the same weights it uses today, until the next training cycle.
At scale, inference is the compute-dominant activity. By 2026, inference accounted for an estimated two-thirds of all AI computing power expenditure (up from one-third in 2023), and 80–90% of the lifetime compute cost of a production AI system (Telnyx, citing Gartner 2026 and industry analysis). See claim-inference-dominant-ai-compute-2026.
Relation to the historical uses of the word
The word inference has been used across classical logic, statistics, and symbolic AI before acquiring this engineering sense. See claim-inference-word-etymology for the Latin root, claim-inference-logical-types for the philosophical lineage, claim-inference-classic-ai-engines for the expert-systems usage, and claim-statistical-inference-meaning for the statistical tradition. The AI engineering sense is the newest and, by 2026, the most economically significant.
See also: claim-llm-inference-prefill-decode, claim-inference-cost-collapsed-280x, claim-test-time-compute-can-substitute-parameters, backpropagation-gap