13B: LLMs Ops

Materials:

Date: Thursday, 07-Nov-2024, 3-4.30pm, IST.

Pre-work:

1. LLM Intro

In-Class

1. Build
   • Building LLMs from Scratch repo
   • pre-training, fine-tuning, instruction fine-tuning
2. Prompt Optimization
   • DSPy, Intro Notebook
3. Structured outputs in LLMs
   • with Instructor
   • with Outlines
   • Blog comparing many libraries from structured output generation.
4. Evaluation
   • with DeepEval
   • with ragas
   • with LangSmith
5. Applications: RAGS
   • Building RAG-based LLM Applications in Production blog, notebook by Gokul Mohandas and Philip Moritz
   • Agentic RAG with LangGraph tutorial
6. Applications: Agents
   • with LangGraph tutorials
   • blog explaining LangGraph
   • Magentic-One

Post-class

• Training with Ray Train
• Serving with Ray Serve, vLLM, ollama for serving
• RAGs with Llamaindex, cognita, Langchain
• Agents with LangGraph, AutoGen, CrewAI, LangGraph IDE blog