---
title: "In AI engineering, 'inference' means running a trained model on new input without updating its parameters"
type: "claim"
status: "seedling"
audit_status: "flagged (V-011)"
date_created: "2026-06-04T00:00:00.000Z"
provenance: "Seek research batch, 2026-06-04"
tags: ["AI","inference","LLM","training","forward-pass","production"]
source_url: "https://telnyx.com/resources/ai-training-vs-inference"
source_title: "AI training vs inference: the 2026 economics split"
source_author: "Telnyx editorial"
source_date: "2026 (exact date not available)"
source_tier: 3
related_notes: ["claim-inference-word-etymology","claim-statistical-inference-meaning","claim-inference-classic-ai-engines","claim-llm-inference-prefill-decode","claim-inference-cost-collapsed-280x","claim-inference-dominant-ai-compute-2026"]
drafted_in: ["2026-07-09-inference-inverted","2026-07-13-we-never-unrolled-it","inference-inverted","we-never-unrolled-it"]
---


In the vocabulary of AI engineering, *inference* designates the stage of a model's life that follows training: the moment when a trained model takes a live input and produces output. The model's [[model weights|weights]] are frozen; no learning occurs. The sole purpose is production: generating a useful result for a user or downstream system.

One formulation captures the distinction cleanly: training is "the one-time, capital-intensive process of teaching a model to recognize patterns across vast datasets," while inference is "the recurring, operational process of actually running that model to answer a question, transcribe a call, or drive a conversation" (Telnyx, 2026).

## The two phases of a model's life

| Phase | What happens | Parameters change? | Frequency |
|---|---|---|---|
| [[Training]] | Model adjusts weights from labelled data | Yes | Occasional; per model version |
| [[Inference]] | Model processes input, returns output | No | Continuous; every user request |

This is, as noted in the Wikipedia entry on [[statistical inference]], an inversion of the statistical sense: a statistician doing Bayesian inference is computing *over* parameters; an AI engineer doing inference is computing *with* frozen parameters. See [[claim-statistical-inference-meaning]] for this divergence.

## What inference looks like in practice

Every user interaction with ChatGPT, Claude, or Gemini is an inference event. When a user submits a prompt, the system runs the model's [[forward pass]] over the tokenized input, producing output one [[token]] at a time. The model is not updated; the weights it used yesterday are the same weights it uses today, until the next training cycle.

At scale, inference is the compute-dominant activity. By 2026, inference accounted for an estimated two-thirds of all AI computing power expenditure (up from one-third in 2023), and 80–90% of the lifetime compute cost of a production AI system (Telnyx, citing Gartner 2026 and industry analysis). See [[claim-inference-dominant-ai-compute-2026]].

## Relation to the historical uses of the word

The word *inference* has been used across classical logic, statistics, and symbolic AI before acquiring this engineering sense. See [[claim-inference-word-etymology]] for the Latin root, [[claim-inference-logical-types]] for the philosophical lineage, [[claim-inference-classic-ai-engines]] for the expert-systems usage, and [[claim-statistical-inference-meaning]] for the statistical tradition. The AI engineering sense is the newest and, by 2026, the most economically significant.

See also: [[claim-llm-inference-prefill-decode]], [[claim-inference-cost-collapsed-280x]], [[claim-test-time-compute-can-substitute-parameters]], [[backpropagation-gap]]