mailwal

Data Pipeline 101: Strategies to make the data pipelines fail... reliably

Vu Trinh

To sustain my work, I’ve opened up paid membership. If you find value in what I do, I’d really appreciate it if you considered upgrading; it helps me keep this work going :d

With only $7/month (billed annually), you can access all the materials you need to grow from junior → senior DE.

Upgrade with 7$/month

  • If you’re a student with an education email, use this 50% ANNUAL DISCOUNT

  • If you’re a Vietnamese user, please DM me for an upgrade due to payment issues. As compensation for the inconvenience, you’ll get 50% OFF the annual plan.

Intro

One of a data engineer's most important jobs is moving data from A to B.

(I’ve said this many times in this newsletter)

In other words: building reliable data pipelines.

But reliability isn't only about successful ones.

In this article, join me as we discuss how to make even failed pipelines…reliable.

Everything can fail

The first strategy is to accept that no matter how well you develop and test, your pipelines (or anything you do in life) can still fail.

Because a pipeline can only run successfully if many factors align. Some you can control; others you can’t:

  • You forget or do badly on things you can control: write bad logic, push no-test code, forget authenticated credentials.

  • You don’t anticipate unexpected things: the SRE team updates the Kubernetes cluster, someone runs an ad-hoc query that consumes all the resources that are supposed to belong to the pipeline runs.

—

The key is not to try to build a perfect pipeline that will never fail. Ensure your work is correct and anticipate incidents; that's still the priority, but it’s pointless to assume that failures won’t happen.

It makes more sense to prepare for failures in a more reliable, controllable way.

Note: This article wont’ discuss how to detect issue via observability methods such as logging, monitoring or tracing. Instead, what I’m gonna discuss is what can we do to make the “fixing“ part more pleasant.

To sustain my work, I’ve opened up paid membership. If you find value in what I do, I’d really appreciate it if you considered upgrading; it helps me keep this work going :d

With only $7/month (billed annually), you can access all the materials you need to grow from junior → senior DE.

Upgrade with 7$/month

  • If you’re a student with an education email, use this 50% ANNUAL DISCOUNT

  • If you’re a Vietnamese user, please DM me for an upgrade due to payment issues. As compensation for the inconvenience, you’ll get 50% OFF the annual plan.

Subscribe to VuTrinh. to unlock the rest.

Become a paying subscriber of VuTrinh. to get access to this post and other subscriber-only content.

A subscription gets you:

Full article repository access + 35-page Spark PDF
Access to learning tools that boost your DE career: dbt, Spark, Airflow, ...
Vietnamese readers contact me for support. You will get 50% off.

© 2026 Vu Trinh
548 Market Street PMB 72296, San Francisco, CA 94104