Published Mar 19, 2024

767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka

Dr. Sebastian Raschka delves into the future of large language models (LLMs), sharing insights on open-source projects like PyTorch Lightning and litGPT that streamline LLM implementation. He introduces cutting-edge training techniques and architectures poised to revolutionize the field, including efficient methods like Lora, Dora, and multi-query attention.

Episode Highlights

Topics covered

Episode Highlights

Related Episodes

733: OpenAssistant: The Open-Source ChatGPT Alternative — with Dr. @YannicKilcher
Answers 383 questions
695: NLP with Transformers — with Hugging Face's Lewis Tunstall
Answers 383 questions
713: Llama 2, Toolformer and BLOOM: Open-Source LLMs — with Meta's Dr. Thomas Scialom
Answers 383 questions
747: Technical Intro to Transformers and LLMs — with Kirill Eremenko
Answers 383 questions
706: Large Language Model Leaderboards and Benchmarks — with Caterina Constantinescu
Answers 383 questions
707: Vicuña, Gorilla, Chatbot Arena and Socially Beneficial LLMs — with Prof. Joey Gonzalez
Answers 383 questions
791: Reinforcement Learning from Human Feedback (RLHF) — with Dr. Nathan Lambert
Answers 383 questions
785: Math, Quantum ML and Language Embeddings — with Dr. Luis Serrano (@SerranoAcademy)
Answers 383 questions
824: Llama 3.2: Open-Source Edge and Multimodal LLMs — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
797: Deep Learning Classics and Trends — with Dr. Rosanne Liu
Answers 383 questions
831: PyTorch Lightning, Lit-Serve and Lightning Studios — with Dr. Luca Antiga
Answers 383 questions
772: In Case You Missed It in March 2024 — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
790: Open-Source Libraries for Data Science at the New York R Conference — with @JonKrohnLearns
Answers 383 questions
678: StableLM: Open-source "ChatGPT"-like LLMs you can fit on one GPU — with @JonKrohnLearns
Answers 383 questions
847: AI Engineering 101 — with Ed Donner
Answers 383 questions

Dexa/Super Data Science: ML & AI Podcast with Jon Krohn

767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka

Topics covered

Popular Clips

Model Size Insights

Simplifying PyTorch Training

Weight Decomposition Insights

Evaluating Language Models

Open Source Contributions

Unlocking LLM Potential

Future Architectures in AI

Data Connect Conference

Decoupling Directions

Community in AI

Multi Query Attention

Mamba vs. Hyena

Reinforcement Learning Insights

Foundations of Machine Learning

Visual Learning Benefits

Episode Highlights

Open-Source Projects

Future of LLMs

LLM Training Techniques

Related Episodes

733: OpenAssistant: The Open-Source ChatGPT Alternative — with Dr. @YannicKilcher

695: NLP with Transformers — with Hugging Face's Lewis Tunstall

713: Llama 2, Toolformer and BLOOM: Open-Source LLMs — with Meta's Dr. Thomas Scialom

747: Technical Intro to Transformers and LLMs — with Kirill Eremenko

706: Large Language Model Leaderboards and Benchmarks — with Caterina Constantinescu

707: Vicuña, Gorilla, Chatbot Arena and Socially Beneficial LLMs — with Prof. Joey Gonzalez

791: Reinforcement Learning from Human Feedback (RLHF) — with Dr. Nathan Lambert

785: Math, Quantum ML and Language Embeddings — with Dr. Luis Serrano (@SerranoAcademy)

824: Llama 3.2: Open-Source Edge and Multimodal LLMs — with Jon Krohn (@JonKrohnLearns)

797: Deep Learning Classics and Trends — with Dr. Rosanne Liu

831: PyTorch Lightning, Lit-Serve and Lightning Studios — with Dr. Luca Antiga

772: In Case You Missed It in March 2024 — with Jon Krohn (@JonKrohnLearns)

790: Open-Source Libraries for Data Science at the New York R Conference — with @JonKrohnLearns

678: StableLM: Open-source "ChatGPT"-like LLMs you can fit on one GPU — with @JonKrohnLearns

847: AI Engineering 101 — with Ed Donner

767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka

Topics covered

Popular Clips

Episode Highlights

Open-Source ProjectsSebastian Raschka explores the capabilities of PyTorch Lightning and Fabric in enhancing PyTorch usability for large language model training. He also introduces the litGPT project, designed to simplify LLM implementations for developers.

Open-Source Projects

Future of LLMsSebastian Raschka, a Staff Research Engineer at Lightning AI, discusses the challenges and innovations in large language model (LLM) development. He explores new model architectures and efficient training techniques that could revolutionize the field.

Future of LLMs

LLM Training Techniques

Related Episodes