Inference
Inference is the process of using a trained model to produce predictions or answers.
Category
AI Inference Stack
These words cover the runtime pieces that shape speed, cost, and throughput when a model answers.
How to recognize this theme
Terms for serving a model efficiently.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
Batching
Batching groups multiple requests so a model can process them together more efficiently.
Quantization
Quantization reduces numeric precision to make model storage and inference more efficient.
Throughput
Throughput is the amount of work a system can process in a given period.
Prompt Template
A prompt template is a reusable structure for shaping model input.
Model Weights
Model weights are the learned parameters a model uses to produce outputs.
Vector Index
A vector index stores embeddings so similar items can be retrieved quickly.
Batch Queue
A batch queue holds work until enough jobs accumulate for grouped processing.