Latency
Latency is the time it takes for a model or system to respond after receiving a request.
Category
Model Serving Ops
These terms cover response time, grouped requests, smaller numeric formats, and cached outputs.
How to recognize this theme
Terms that describe how AI systems answer requests efficiently.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
Batching
Batching combines multiple inputs so a system can process them together more efficiently.
Quantization
Quantization reduces the precision of model numbers to save memory and speed up inference.
Cache
A cache stores previous results so repeated requests can be served faster.