Batching
Batching combines multiple requests so shared hardware can process them more efficiently.
Category
GPU Throughput
These terms describe common ways to make inference faster, denser, and more efficient on shared hardware.
How to recognize this theme
Terms that reduce latency and raise model capacity.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
KV Cache
A KV cache stores prior attention keys and values so generation can continue faster.
Quantization
Quantization reduces numerical precision so models use less memory and often run faster.
Tensor Parallelism
Tensor parallelism divides model computation across multiple devices to increase usable throughput.