You can’t buy taste. We tested it.

Which model has the best taste in website design?

We gave ten foundation and open models the same ten creative briefs, then asked people to choose between the results. The priciest model didn’t run away with it. And labs that never compared notes kept landing on the same design choices.

If your product generates creative work, or depends on innately subjective human evaluation, your users may be getting the same output as your competitor’s. Better prompts won’t fix it. Your benchmarks won’t flag it.

Join HumanSignal ML Researcher Micaela Kaplan and Creative Director Todd Morey, who wrote the briefs, on Thursday, October 29 at 11am ET / 8am PT for You Can’t Buy Taste, a 30-minute live session on evaluating generative AI with human judgment.

You’ll learn:

• Why capability benchmarks can’t rank creative work

• Why more inference spend doesn’t buy better taste

• Why annotator disagreement is a finding, not noise

• Why we didn’t use an LLM judge

Bring your questions. Micaela and Todd are taking them live.

Can’t make it live? Register and we’ll send you the recording.