Inference
Inference is the process of using a trained model to produce an output.
Category
Model Serving
These words describe how inference systems trade speed, cost, and precision.
How to recognize this theme
Terms used when a model answers quickly and efficiently.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
Batching
Batching combines multiple requests so a system can process them more efficiently.
Quantization
Quantization reduces numeric precision to make models smaller or faster.
Throughput
Throughput is the amount of work a system can process per unit of time.
Batching
Batching groups requests together so a model can process them more efficiently.
Quantization
Quantization uses lower-precision numbers to make a model smaller or faster.
Latency
Latency is the delay between a request and a model response.
Throughput
Throughput is the number of requests or tokens a system can handle per unit time.
Batching
Batching groups requests together so a model can process them more efficiently.
Quantization
Quantization uses lower-precision numbers to make a model smaller or faster.
Latency
Latency is the delay between a request and a model response.
Throughput
Throughput is the number of requests or tokens a system can handle per unit time.
Inference
Inference is the process of using a trained model to generate outputs for new inputs.
Batching
Batching combines multiple requests so they can be processed more efficiently.
Quantization
Quantization reduces numerical precision to make models smaller and faster to run.
Latency
Latency is the delay between a request and the start or completion of a response.