Tokenization
Tokenization is the process of splitting text into units a model can read and process.
Category
Model Inference
These words describe how text is split, scored, and constrained during generation.
How to recognize this theme
Terms used when a trained model turns input into output.
In a daily board, this category groups terms by their shared role. Look for four cards that describe the same mechanism, risk area, or workflow rather than four words that merely sound similar.
Educational context
These entries are vocabulary notes for learning. They are not project endorsements, token recommendations, exchange rankings, or trading signals.
Attention
Attention is a mechanism that lets a model focus on the most relevant parts of its input.
Context Window
A context window is the maximum amount of input and generated text a model can consider at once.
Hallucination
A hallucination is a model output that sounds plausible but is not grounded in the provided information.