Data Elixir - Issue 594

The 100+ page guide to orchestrating LLM workflows*

Your LLM workflow works in a notebook. Now it needs scheduling, retries, monitoring, and a way to ask for human review. This 100+ page guide shows how Apache Airflow brings those pieces together, with practical patterns and code examples for LLM tasks and agents. Free to download.
*Message from this week's sponsor, Astronomer

Fifty years of the data stack

Today’s data stack makes more sense when you see how we got here. This interactive timeline traces 50 years of analytical systems, showing how database access, query engines, and storage evolved together, and why older technologies still sit alongside newer ones.
Columnar | Ian Cook

How to be more ambitious: A tutorial for humans with models

Most people use AI to do familiar tasks faster, but Felix Rieseberg argues we're thinking too small. In this post, he explores three ways to be more ambitious, with practical advice and concrete examples.
Felix Rieseberg

Can you use autoregressive diffusion to generate market data?

Jane Street tried using diffusion models to generate realistic streams of stock market events but it turns out that the market data's mixture of continuous values and discrete actions makes this difficult. This research write-up explores what worked, what didn't, and what the experiments revealed about generative modeling.
Jane Street Blog | James Somers

Better Context, Better Answers

Better business context can matter more than a better model. In AtScale’s preliminary benchmark results, adding a semantic layer improved accuracy more than switching models. Join AtScale and analyst Andrew Brust on October 28 at 2 pm ET to examine the comparisons and learn how to test whether AI gets your business numbers right.
// sponsored

Dogs of New York

nycdogs is an R package containing New York City's dog licensing and dog bite data. It's great for teaching because the datasets are large, messy, and full of interesting things to explore. Besides, everyone loves dogs. In this post, Kieran Healy explores the data and shows how duplicate records, messy zip codes, and missing information can complicate even simple questions.
Kieran Healy

Dual axes, not dueling

Every dual-axis chart has an implied exchange rate between its two scales, and that choice can change the story the chart tells. In this post, RJ Andrews uses animated examples and historical charts to explain when dual axes are misleading, when they're useful, and what works better.
dbt Charts | RJ Andrews

30 Months on the Exponential (in Charts)

“This is the worst AI you'll ever use.”
One Useful Thing | Ethan Mollick

Python 3.15 for Data Scientists

Python 3.15 is here, with some useful improvements for data scientists. This practical walkthrough covers lazy imports for faster startup, immutable dictionaries for safer configurations, and a new profiler that can find bottlenecks in running code. Includes examples, benchmarks, and visualizations.
Khuyen Tran

Frisket

Frisket is an open-source AI spreadsheet that turns unstructured information into data you can easily work with. Load PDFs, emails, audio, or video, then extract, classify, and analyze the contents in a familiar spreadsheet interface. It supports local AI models, so you can even run the whole thing privately.
Jonathan Soma

Tiny Charts: Data Viz as Postage Stamp Art

This is a great deep dive into how countries use data visualization to communicate national priorities, economic growth, and scientific progress on... postage stamps. There are 180 examples here spanning decades of this unusual genre, along with a spreadsheet of the entire collection.
Nightingale | Julie Peasley