mailwal

I tested Haiku for a week.

Learn Analytics Engineering

I tested Haiku for a week.

An experiment using Anthropic’s “dumbest” model for my data tasks- what tasks is it best for and how much money does it actually save?

Oct 22

When I first started using Claude Code in my work, everyone would rave about the latest, smartest models- Sonnet, Opus, Fable, Mythos. It was always about how much they could do and understand.

Now I’m seeing a shift away from these models. Don’t get me wrong, they are still powerful. But more people are realizing they don’t actually need to use AI agents for the most complex data tasks.

Sure, Claude can really help with planning a data model and weighing different design decisions, giving you a high-level overview of the tradeoffs. However, when it comes to implementing the suggestions, it’s often easier to implement them yourself. You don’t have to worry about:

  • running through a sh*t ton of tokens

  • reading through long AI-jargon summaries

  • continually course-correcting the model and reverting work when it oversteps

I’m trying to find a balance between what is worth partnering with Claude’s highly intelligent models on, what’s worth off-loading to the “dumber” models, and what’s worth doing myself.

If you don’t already know, Haiku is Anthropic’s fastest and least expensive model. They recently released Haiku 5.5, naming it their “cheapest, fastest, and most capable small model” they’ve ever released. It performs over twice as well as the previous Haiku model and is 10-50% of the cost.

This sounds promising.

I want to see exactly what Haiku is capable of, how many fewer tokens it uses, and how much time it saves me on simple, monotonous tasks.

There are a few specific tasks I want to test it on. These include:...

User's avatar

Continue reading this post for free in the Substack app

© 2026 Madison Mae
548 Market Street PMB 72296, San Francisco, CA 94104