Startseite
Podcasts
Signals and Threads Podcast
Building a data warehouse from scratch with Jacob Baskin

Building a data warehouse from scratch with Jacob Baskin

Signals and Threads vor 3 Monaten

0:00

1:34:29

In university Jacob Baskin studied at the intersection of
computer science and economics, thinking about systems that
incentivize people to express their true preferences. He put
those ideas into practice at Google, where he worked on ad
serving, before joining Jane Street’s database infrastructure
team. In this episode, Ron and Jacob discuss Superstore, a
distributed columnar database now central to Jane Street’s tech
stack that Jacob began building practically the day he started.
How do you support wide-ranging analytical queries while
transactional writes stream in at the speed of trading systems?
And what’s it like when your first design doc leads to an
eight-figure hardware purchase? After building Superstore Jacob
has since gone back to his roots, thinking about schemes for
bidding on compute time as he works to optimize usage of the
Hive, Jane Street’s massive compute cluster for research.

You can find the transcript for this episode on our website.

Some links to topics that came up in the discussion:

Mechanism design, second-price auction

MapReduce, BigTable, Google File System

Vertica

Apache Parquet

CockroachDB

Paxos

BitTorrent