Latency
Latency is the delay between a request and the start or completion of a response.
Category
Inference Serving
These words describe the runtime tradeoffs that shape model responsiveness and serving efficiency.
How to recognize this theme
Terms you see when model output speed and cost matter.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
Batching
Batching processes multiple inputs together to improve hardware efficiency.
Throughput
Throughput is the amount of work a system completes over a given time period.
Quantization
Quantization reduces numeric precision to make model serving faster or cheaper.