The End of Data Leakage

Industrial AI Podcast vor 4 Monaten

Why Data Leakage Threatens AI Benchmarks—and How We're Fixing It. Discover the future of reliable model evaluation.

0:00

24:06

Podcast

Industrial AI Podcast

350 Episoden

How the industry can benefit from AI technologies and what it takes to do so.

Wissenschaft Technologie Wirtschaft

Podcast aneignen

In this episode, we dive deep into the challenges facing time
series AI model leaderboards, from hidden information leakage to
the complexities of benchmarking foundation models. I sit down with
Marcel Meyer to unpack why traditional approaches fall short and
how our new TS Arena leaderboard is setting a new standard for
fair, future-proof evaluation. We explore the pitfalls that plague
current benchmarks, the surprising ways data contamination can skew
results, and the innovative pre-registration protocol we've
developed to keep evaluations honest. If you've ever wondered what
it takes to build a truly trustworthy AI leaderboard—or why it
matters for industry and research alike—this conversation is packed
with insights you won't want to miss.