Inference

Inference is what happens every time someone uses a trained AI model: a request goes in, and the model computes a response. It is distinct from training, which builds the model in the first place.

Inference matters because it is an ongoing cost that grows with use. Each chatbot answer, generated image, or agent step uses computing power and electricity. Models that "think" longer before answering, or agents that take many steps, use more inference per task.

Claims about a model being cheaper or more efficient usually refer to inference costs. Check whether a figure is per request, per token, or for the whole system.