Published Apr 28, 2023

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation) — with Jon Krohn

Jon Krohn delves into Low-Rank Adaptation (LoRA) as a powerful method for parameter-efficient fine-tuning of large language models, highlighting its ability to reduce computational demand and prevent catastrophic forgetting, thereby democratizing access to advanced AI technologies.

Episode Highlights

Topics covered

Popular Clips

Efficient Model Training
Play Clip

Episode Highlights

Related Episodes

704: Jon’s “Generative A.I. with LLMs” Hands-on Training — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
670: LLaMA: GPT-3 performance, 10x smaller — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
692: Lossless LLM Weight Compression: Run Huge Models on a Single GPU — with Jon Krohn
Answers 383 questions
678: StableLM: Open-source "ChatGPT"-like LLMs you can fit on one GPU — with @JonKrohnLearns
Answers 383 questions
728: Use Contrastive Search to get Human-Quality LLM Outputs — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
788: Multi-Agent Systems: How Teams of LLMs Excel at Complex Tasks — with @JonKrohnLearns
Answers 383 questions
797: Deep Learning Classics and Trends — with Dr. Rosanne Liu
Answers 383 questions
694: CatBoost: Powerful, efficient ML for large tabular datasets — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
772: In Case You Missed It in March 2024 — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
824: Llama 3.2: Open-Source Edge and Multimodal LLMs — with Jon Krohn (@JonKrohnLearns)
Answers 383 questions
706: Large Language Model Leaderboards and Benchmarks — with Caterina Constantinescu
Answers 383 questions
784: Aligning Large Language Models — with Sinan Ozdemir
Answers 383 questions
767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka
Answers 383 questions
778: Mixtral 8x22B: SOTA Open-Source LLM Capabilities at a Fraction of the Compute — with Jon Krohn
Answers 383 questions
SDS 510: Deep Reinforcement Learning — with Jon Krohn
Answers 383 questions

Dexa/Super Data Science: ML & AI Podcast with Jon Krohn

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation) — with Jon Krohn

Topics covered

Popular Clips

Efficient Model Training

Episode Highlights

LoRA Introduction

Parameter-Efficient Fine-Tuning

Related Episodes

704: Jon’s “Generative A.I. with LLMs” Hands-on Training — with Jon Krohn (@JonKrohnLearns)

670: LLaMA: GPT-3 performance, 10x smaller — with Jon Krohn (@JonKrohnLearns)

692: Lossless LLM Weight Compression: Run Huge Models on a Single GPU — with Jon Krohn

678: StableLM: Open-source "ChatGPT"-like LLMs you can fit on one GPU — with @JonKrohnLearns

728: Use Contrastive Search to get Human-Quality LLM Outputs — with Jon Krohn (@JonKrohnLearns)

788: Multi-Agent Systems: How Teams of LLMs Excel at Complex Tasks — with @JonKrohnLearns

797: Deep Learning Classics and Trends — with Dr. Rosanne Liu

694: CatBoost: Powerful, efficient ML for large tabular datasets — with Jon Krohn (@JonKrohnLearns)

772: In Case You Missed It in March 2024 — with Jon Krohn (@JonKrohnLearns)

824: Llama 3.2: Open-Source Edge and Multimodal LLMs — with Jon Krohn (@JonKrohnLearns)

706: Large Language Model Leaderboards and Benchmarks — with Caterina Constantinescu

784: Aligning Large Language Models — with Sinan Ozdemir

767: Open-Source LLM Libraries and Techniques — with Dr. Sebastian Raschka

778: Mixtral 8x22B: SOTA Open-Source LLM Capabilities at a Fraction of the Compute — with Jon Krohn

SDS 510: Deep Reinforcement Learning — with Jon Krohn

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation) — with Jon Krohn

Topics covered

Popular Clips

Episode Highlights

LoRA IntroductionJon Krohn explores Low-Rank Adaptation (LoRA) as a method for efficient fine-tuning of large language models. He discusses how LoRA addresses issues like catastrophic forgetting and reduces computational demands, making advanced AI accessible to more users.

LoRA Introduction

Parameter-Efficient Fine-Tuning

Related Episodes