Probability Distributions Explained
Kirill delves into the intricacies of probability distributions within transformers, highlighting how 200,000 values represent the likelihood of each word in the English language. He explains the process of generating multiple probability distributions for words, emphasizing that only the last distribution is utilized during inference. This efficient approach showcases the balance between computational demand and practical application in training AI models.In this clip
From this podcast
Super Data Science: ML & AI Podcast with Jon Krohn
747: Technical Intro to Transformers and LLMs — with Kirill Eremenko
Related Questions