🛠️ Event-driven workflows with Airflow | DataTalks.Club Weekly
#300 · October 26, 2026
This week: an Astronomer webinar on event-driven Airflow, the Project 1 deadline for AI Dev Tools Zoomcamp, Module 6 of Machine Learning Zoomcamp, two events on November 2, a podcast recording on production ML, and Alexey's post on agentic engineering.
Build event-driven workflows that scale with Airflow
What if you could trigger Dags from Kafka messages?
Join Airflow expert Marc Lamberti on Nov 4th as he demos Airflow 3 pipelines that trigger the instant an event arrives and how to run them at scale with your data.
Don't miss how to:
Build and operate event-driven pipelines
Trigger Dags based on messages from Kafka, Amazon SQS, etc.
Run many watchers on one source without dropping events
New Free Course: 2026 Cohort Registration is Open
What happens when your agent can launch expensive, slow, failure-prone GPU jobs?
In this free course for AI/ML and MLOps engineers, you build a Personalized Media Factory: an agent writes copy, generates images plus voice or video, and regenerates on failure.
Run media models on serverless GPUs, no cluster to manage
Scale from 1 to 1,000+ records with a cost line per asset
Add evals, tracing, and SLOs
Approved applicants get Nebius Builders Program credits. 44 people have already registered.
Project 1
Last call for Project 1: submissions close on October 27, 2026 at 00:00 CET (October 26 at 23:00 UTC). Check what to build and how it's scored before you submit.
After the deadline, you review three projects from other participants. Reviews are part of the project: without them, your project doesn't pass. They're due November 3, 2026 at 00:00 CET (November 2 at 23:00 UTC).
Not ready yet? You can submit on the second attempt, Project 2, until November 17, 2026 at 00:00 CET (November 16 at 23:00 UTC).
Module 6: Decision Trees and Ensemble Learning
Linear models are a good start, but tree-based models often work better on tabular data. In Module 6, we build a credit risk scoring model and combine many trees into stronger models with random forests and gradient boosting with XGBoost.
The deadline for Homework 6 is November 3, 2026 at 00:00 CET (November 2 at 23:00 UTC).
Still finishing Homework 5? There's still time: you can submit it until October 27, 2026 at 00:00 CET (October 26 at 23:00 UTC).
After this module, we start the midterm project.
EVENTS
Upcoming Events
MON
2
NOV
MON
2
NOV
RECORDINGS
Podcast Recording: Building AI Evaluations You Can Actually Trust
On October 12, we talked with Miguel Otero Pedrido, ML AI Engineer at Zapier and founder of The Neural Maze, about building ML and AI systems for production. We discussed:
When traditional ML still beats LLMs
Solving the cold start problem with contextual bandits for recommenders
Avoiding comprehension debt as ML shifts toward AI system design
Trace redaction pipelines and error analysis on agent traces at Zapier
If you missed the live session, the recording is ready.
READ
From Alexey on Data: Trends in Agentic Engineering
Do coding agents write more of your code than you can review?
In his new newsletter, Alexey looks at agentic engineering, following his webinar with Hugo Bowne-Anderson:
The implementer-reviewer approach: one agent builds, another tries to reject the result
Books like Clean Code as shared rules for agents
Organizing agents into teams and larger hierarchies
Dark software factories, where humans never look at the code
DataTalks.Club is a global community of data enthusiasts. If you haven't had a chance to join our Slack yet, use this invite link. If you have problems with the invite link, fill in this form, and we'll add you to Slack manually.
Do you have any questions? Hit reply, and we'll be glad to answer.
See you in Slack!
Copyright (C) 2026 DataTalks. All rights reserved.
You're receiving this email because you registered in our Slack community DataTalks.Club or attended one of our events
Our mailing address is:
DataTalks Langhansstr. 70 Berlin 13086 Germany
Want to change how you receive these emails?