13A: LLMs Introduction

Materials:

Date: Thursday, 07-Nov-2024, 11.30-1pm, IST.

Pre-work:

1. LING571@UW Deep Learning For NLP, Prof. Shane at UW, Spring’24. Introduction, Word Vectors, Language Modeling
2. CIS7000@UPenn LLMs, by Prof. Mayur Naik at UPenn, Fall’24, Background, Language Modeling
3. AIL821@IIT-Delhi LLMs: Introduction and Recent Advances ELL881/AIL821, LLMs: Introduction and Advances @ IIT-Delhi, Fall’24.
4. Transformers
   • LLMs @ UPenn Part-1, Part-2
   • LLMs: Introduction and Recent Advances @ IIT Delhi Module-5 on RNNs, Module-6 on Attention and Transformers

In-Class

We will follow the “Follow the data” approach to organize the content.

1. Quick review of NLP and Deep Learning for NLP, pre- and post-GPT world.
   • Lecture 2 from AIL821 Introduction to NLP,
   • Lecture 3.1 from AIL821 Introduction to Language Models
2. LLM Flow: (Quality) Datasets, Model Training (Pre-training, Alignment, Fine-tuning), Prompt Optimization, Constrained Language Generation, Evaluation.
3. Datasets and Tasks (to train LLMs)
   • Lecture 7 from AIL821
4. Model Training
   • Pre-training
     • Lecture 12.1 from AIL821
   • Alignment
     • Direct Preference Optimization: Your Language Model is Secretly a Reward Model
   • Fine-tuning
     • LoRA: Low-Rank Adaptation of Large Language Models
5. Prompt Optimization
   • Chain-of-Thought
6. Constrained Language Generation
   • Collection
7. Evaluation
   • Scaling Evaluation of LLMs Yann Bubois, CIS 7000 LLM Course
8. Applications and Design Patterns
   • Tools
     • Gorilla
   • Agents
     • Lilian Wang’s blog on LLM Powered Autonomous Agents
     • Aman’s blog on Agents
   • RAG
     • Paper from NVidia FACTS About Building Retrieval Augmented Generation-based Chatbots
9. LLMs can not reason & plan
   • LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks

Post-class

1. Datasets and Tasks (to train LLMs)
   • LIMA: less is more for alignment
   • Instruction Tuning for Large Language Models: A Survey
   • OLMo @ Allen AI - if you are interesting in all aspects of open-source LLM development.
2. Model Training
   • Pre-training
     • Improving Language Understanding by Generative Pre-Training
   • Alignment
     • Lecture 12.2 from AIL821
   • Fine-tuning
     • Performance Efficient Fine-Tuning collection
     • Lecture: PEFT
     • Lecture: : Quantization and Pruning
     • QLoRA: Efficient Finetuning of Quantized LLMs
3. Prompt Optimization
   • The Prompt Report
   • Chain-of-Thought
   • Tree-of-Thought
   • Self-Reflection
   • Self-Contrast
   • Think before you Speak
4. Constrained Language Generation
   • collection
   • Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation
5. Evaluation
   • Scaling Evaluation of LLMs Yann Bubois, CIS 7000 LLM Course
6. Applications and Design Patterns
   • Tools
     • [Lecture 18.2 from AIL821] LLMs and Tools: Function Calling
   • Agents
     • [Lecture 18.3 from AIL821] LLMs and Tools: Agentic
     • AutoGen repo
     • CrewAI repo
     • LLM Agent papers collection
     • Survey: The Rise and Potential of Large Language Model Based Agents: A Survey
   • RAG
     • Retrieval-Augmented Generation for Large Language Models: A Survey Mar’24
     • Searching for Best Practices in Retrieval-Augmented Generation Jul’24
7. LLMs can not reason & plan
   • Lecture 19 from AIL821 Reasoning in LLMs
   • GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
   • Learning to reason with LLMs
   • Systems-2 Collection Repo

LLMs and Influence Functions

1. Studying Large Language Model Generalization with Influence Functions
2. Do Influence Functions Work on Large Language Models?
3. TextGrad Automatic ‘’Differentiation’’ via Text, paper

Full Courses

1. CIS7000 LLM Course @ UPenn by Prof. Mayur Naik. Covers many advanced topics.
2. AIL821 LLMs Course @ IIT-D
3. Deep Learning For NLP @ UW LING 574, Deep Learning For NLP, Prof. Shane @ UW, Spring’24.
4. Walk through the book Building LLMs from Scratch