mailwal

Data Berlin #59

mucio from the Data Berlin meetup

Data Berlin #59

The tasteful and stylish issue, like the grey sky over Berlin

Nov 2

Brace yourself: winter is coming, and the forecast bottoms out at zero by Sunday. Time to find out where you stored your scarf. Which is also a reminder that you can optimize your bucket costs with cold storage. We are trying to see the positive side here.

It’s late to invite you to our next meetup — it’s tomorrow — but the stream is on our YouTube channel from 19:00, and the talks go up a few days later.

Two months left in 2026: some people are still building things, others spent the last few months writing about what they built. We enjoyed collecting these. We hope you enjoy reading them.

No further ado: close the open windows and let’s go.

Data Berlin Newsletter is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

📖Interesting Readings

Fivetran swallowed dbt, IBM bought Confluent, SAP bought Dremio, and Joe Reis borrows a line from Gramsci to describe an industry selling a future it has not arrived in while practitioners keep the present running. – The Great Data Interregnum - Morbid Symptoms of an Industry Between Eras

Four families of agent builders are heading in different directions, and this essay traces where each came from and why the middle of the stack lost its footing, with the author declaring what he sells before he starts. – The Lost Canvas: How the middle of the agent stack is getting crushed

Two 2.0 releases landed within weeks of each other, and someone who has run both in production for years compares what actually changed for the people writing data code every day. – Polars 2.0 vs DuckDB 2.0

Scout24 built its own data agent for Slack rather than buying one, and explains what had to be true of the catalog and the evaluations before stakeholders started trusting an answer. – Beyond the Chat Window: How We Built a Data Agent Our Stakeholders Can Trust

Ben Lorica changes his mind about world models after a robotics result, and works through what that shift implies for the data that trains and feeds these systems. – I changed my mind about world models.

Postgres 19 is late, and a core committer explains why: far more bugs are being found than a year ago, back-patches jumped by more than 60 percent, and humans still have to read every proposed fix. – Postgres 19 is late, and it's AI's fault

Why are the strongest developers the slowest to embrace coding agents? A Thoughtworks technologist argues the hesitation is not nostalgia: built and working in production are two very different claims. – Why are good developers so wary of agentic development?

An autonomous agent has been fixing ClickHouse's broken builds since March, opening its own issues and pull requests, and the maintainers describe what it takes to hand a machine the unglamorous half of maintenance. – Meet GroeneAI: our autonomous contributor to ClickHouse

🚀 Shipping Now

  • dbt-hodor — A free, self-hosted take on dbt's new paid state service: during a run it reuses every model whose logic and upstream data are unchanged, keeping that state in Delta, Trino, Postgres or a local DuckDB file.

  • Snowflake secondary indexes — A table can now carry more than one index mapping values to the micro-partitions that hold them, so selective lookups and range filters skip data instead of scanning, with maintenance billed through the search optimisation service.

  • Databricks real-time pipelines — Real-Time Mode reaches general availability inside declarative pipelines, bringing millisecond latency to a team's streaming and ETL definitions without keeping a second engine alongside them.

  • BigQuery Arrow results — Query results can come back in the columnar Arrow format, removing a serialisation step for anything feeding a dataframe, a notebook or another analytical engine. Preview.

  • MotherDuck Dive Apps — A chart in the warehouse can now trigger serverless Python jobs that write back into source systems, send messages or rerun models, so a dashboard becomes an app an agent can assemble.

  • MiniLake — A local emulator of the Databricks API that executes SQL on DuckDB, writes real Delta files and runs jobs in a container, so the CLI, Terraform and agents work on a laptop and air-gapped.

☕ Upcoming Meetups & Events

💼 This Week’s Job Picks

📊 Data Analyst

📈 Analytics Engineer

⚙️ Data Engineer

🤖 AI / ML / LLM + Data Scientist

🧭 Leadership

Are you still here? Then: two events this month. If you spotted the two bears in the list, you already know we are running a build night on the 25th of November.

We are also planning the next one for December.

It’s time to press the send button
— Data Berlin team

Data Berlin Newsletter is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

There is absolutely nothing after this line, but in case you are curious and you want to support us, feel free to click.

Absolutely nothing.

Invite your friends and earn rewards
If you enjoy Data Berlin Newsletter, share it with your friends and earn rewards when they subscribe.

Invite Friends

© 2026 mucio