SE Radio 610: Phillip Carter on Observability for Large Language Models

Phillip Carter, Principal Product Manager at
Honeycomb and open source software developer, talks with host
Giovanni Asproni about observability for large language models
(LLMs). The episode explores similarities and differences for
observability with LLMs versus more conventional systems. Key
topics include: how observability helps in testing parts of LLMs
that aren't amenable to automated unit or integration testing;
using observability to develop and refine the functionality
provided by the LLM (observability-driven development); using
observability to debug LLMs; and the importance of incremental
development and delivery for LLMs and how observability
facilitates both. Phillip also offers suggestions on how to get
started with implementing observability for LLMs, as well as an
overview of some of the technology's current limitations. This
episode is sponsored by WorkOS.