154. Token
A unit of data, such as a word or subword, used as the basic building block for processing and generating text in language models.
Last updated
A unit of data, such as a word or subword, used as the basic building block for processing and generating text in language models.
Last updated