Prefill
Prefill is the initial pass that processes the input prompt before generation begins.
Category
Serving Pipelines
These terms describe the steps from prompt processing to token streaming and traffic management.
How to recognize this theme
Terms that show how model requests move through production systems.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
Token Streaming
Token streaming delivers model output incrementally instead of waiting for the full answer.
Request Routing
Request routing chooses which service or worker handles a given model request.
Autoscaling
Autoscaling changes available capacity based on demand so a system can keep up with traffic.