Your team vibe-coded an app. When do you need a real engineer?

Road To Carry NewsletterNov 4, 2026

Your team vibe-coded an app. When do you need a real engineer?

What to check on vibe-coded apps

We will come up with a new logo in the future!

Note from Edwin: Hi all - today, we have a new guest writer for the AI To Carry series! Hasan is a former AI engineer at Microsoft AI and a Research Engineer at Citadel Securities (yes, his engineering chops are very legit). He touches on a topic that I think many PE investors and portcos are struggling with, i.e., how do you go from vibe coding to production grade AI tools that drive actual P&L impact?

“I’ll vibe code the CRM for us.”

I’ve heard some version of that line in more than one conversation with non-technical professionals this year. The other classic is “I want to roll this out to the entire company next week,” usually said about an app that has had exactly one user so far: the person who vibe-coded it over a weekend.

To be clear, we love that people are building. An ops lead can now make a working tool in a day (that's what "vibe coding" means: describing what you want to an AI and letting it write the software).

But an app that works in a demo is not the same as a production-grade app that drives real EBITDA improvement. Between the two sits a set of unglamorous problems, and if nobody fixes them, the app that was supposed to save money ends up costing money (e.g., a customer data leak, a billing error, or a week of the CFO apologizing to the top accounts).

This post is for the investor or operator who has to decide which of these apps to trust, which ones can hurt the P&L, and who should own them.

If you’re an AI or tech provider serving the PE ecosystem interested in supporting AI To Carry series with sponsorships, please reply to this email with your company website and budget!

Several PE and indy sponsor backed portfolio companies in the Road To Carry network have worked with Chariot Labs, and the feedback I’ve heard has been very positive, with all engagements leading to additional projects. So if you’re interested in how AI can be embedded into portco to drive real value creation, reach out!

Chariot Labs helps PE-backed companies identify and prioritize where AI can create value, then embeds AI engineers to build and deploy those workflows into their existing systems.

Founded by former PE investors and AI engineers, we specialize in business services, industrial services, and multi-site retail. Our engagements range from field services companies to education services, and include work within the Apax and Stirling Heights portfolio.

Want to know how your competitors are using AI? Book a free call with the Chariot Labs founding team today.

Learn More

State of Play

Hi, I'm Hasan, cofounder of Chariot Labs, subbing in for Davis this week. I've spent a lot of time taking vibe-coded apps and making them safe and scalable for a business to rely on.

My working view as of November 2026: with current models, it's very easy to vibe code an app that works 90% of the time. The last 10% is where all the liability sits: backups, who is allowed to see what, how updates go live, and simple questions like "what happens when two people click the same button at once?"

For example, after a security incident at Railway (a third party provider that hosts apps on the internet), I helped a client change all of their passwords and access keys. It should have taken 15 minutes, but it took far longer, because the same keys were reused across several apps, so every change broke something else. Nobody had a list of which app used what and basic engineering principles for security simply did not exist.

Unfortunately, this is pretty common. When RedHunt Labs scanned about 130,000 sites built on vibe-coding platforms, roughly 1 in 5 leaked at least one secret (a password or key that unlocks a database, a payment account or another system).

When a vibe-coded app breaks, it doesn't show up as an IT ticket. It shows up as a breach notice to customers, a stack of credit memos, or an account that doesn't renew.

Waiting for “better AI” won't fix it. Veracode tested more than 100 AI models on 80 coding tasks, and 45% of the code they wrote had a well-known security hole.

Recent public failures didn't take a clever hacker. They were mainly caused by somebody skipping the basics because the apps simply “looked finished”:

Lovable. Apps built on this platform shipped without basic rules about who can see which data. A researcher found 170 of them where anyone could pull customers' names, emails and payment status (CVE-2025-48757), and one app on Lovable's own showcase exposed about 18,000 users (The Register, Feb 2026).
Replit and SaaStr. During Jason Lemkin's SaaStr experiment, Replit's AI assistant deleted a live database, while under instructions not to change anything. It then said the data couldn't be restored, which turned out to be wrong (Fortune, July 2025).

OK, enough horror stories.

The first question I get from vibe-code builders at this point is: "Why can't I just tell Claude Code to make it safe?"

Quick answer: you definitely should, and it helps.

But the AI fixes what you ask about, and it may not give you the right answer. The hard part is not only knowing what to ask but also ensuring you have designed for both the (i) “everything goes well” path as well as the (ii) path where “everything goes wrong.” Those questions and design decisions come from having watched apps break in production, and that's most of what a technical review is.

Here's a practical step-by-step on how to decide which apps deserve that technical review and engineering resources.

Where to focus? Which apps can hurt the P&L

You should start with the ones where a mistake costs the most money or creates the most liability. I sort them into three levels:

Read-only (e.g., a dashboard, a contract search tool, a KPI summary). If it's wrong, someone reads a bad number and hopefully double-checks it. Overall, low risk.
System of record (i.e., where the official numbers live: the customer list, the job history, the inventory count). If it's wrong, every decision based on it is wrong too, and you might not find out until the quarter closes.
Acting on your behalf (i.e., the app does things in the real world: sends invoices, places orders, moves money, emails customers, often with no undo button). A mistake here reaches customers and cash before anyone sees it.

Our view is that once an app reaches Level 2, critical business operations depend on it. That's the point to stop adding features, get a proper technical review, and treat every change after that like a change to any other core system.

Example: the invoice app

Picture a commercial HVAC services portco where two AR (accounts receivable) clerks retype each job ticket into an invoice, usually about five days after the job closes. The controller gets fed up and vibe-codes an app over a weekend: when a technician closes a job, the app builds the invoice and emails it to the customer.

The upside is real:

- At 1,200 jobs a month and a $1,500 average ticket, the portco bills $1.8M a month.
- Billing five days sooner pulls roughly $300K of cash forward, a one-time working-capital gain, as long as customers keep paying on the same terms.
- The two clerks move to collections.

That's a Level 3 app (i.e., acting on your behalf), and here's what can go wrong with "send the invoice":

The customer gets billed twice. A technician's phone loses signal, and when it reconnects, the app thinks the job closed twice.
A $980 job becomes a $9,800 invoice. The tech types 40 hours instead of 4.0, and the app does the math perfectly on the wrong number.
Customer A sees Customer B's invoice. Change one digit in the invoice link and you're reading another customer's pricing.
Invoices stop going out and nobody notices. The email service starts rejecting messages, but the screen still says "sent."
Every customer gets last month's invoices again because a bug re-ran old jobs.
Nobody can explain a disputed charge, because nothing recorded which job, hours or rate produced it.

The demo probably catches none of these. The controller probably tests one job with clean data, watches that one invoice arrive successfully, and rolls it out.

Take #4 from above (i.e., invoice stops sending). Three weeks of silent failures leave roughly $1.35M of completed work unbilled. That's:

Cash arriving weeks late,
A dent in the month's collections, and
More importantly, a batch of confused customers when the backlog finally goes out.

Practical AI for PE Breakfast & Learn Workshops

Want hands-on AI training built specifically for private equity?

Thanks everyone for coming out to the AI for PE breakfast & learn pilot series. We’ve had over 150 PE professionals, ranging from founders/group heads to senior associates and associates. Based on amazing feedback, we will be running these more regularly after a short break!

Thanks for joining us!

How to implement with the right guardrails

The obvious fix is to have AR review every invoice before it goes out, but then within two weeks you’re back to the five-day lag, and the controller’s vibe coded app has become just a redundant version of the old process.

A better setup would be to:
- Keep same-day billing for routine jobs and slow down only the risky ones.
- For example, standard jobs under $5K go out automatically, and anything larger, or anything where a tech changed the hours or the rate, lands in a queue AR clears once a day.

Underneath that, a few engineering habits cover most of the rest of the list:

Each invoice gets one ID tied to the customer, job and billing period, so a phone that reconnects to the web application five times still produces one invoice.
Inputs get checked against what’s expected, which is how 40 hours on a four-hour job gets caught.
An invoice is only marked sent once the email service confirms it, and AR hears about failures the same day.
A volume cap and a log of every invoice’s job, hours, rate and send result make a runaway batch much less likely and a dispute much faster to settle.

But this is all getting to the technical nitty-gritty…

What to do as an operating partner or executive?

Here’s a checklist you can implement today. Sit down with the vibe-code builder for 30 minutes and run through this four-part checklist (every item is a yes or a no question):

1. Could it leak data or money?

Passwords kept out of the app. No passwords or keys are written into the app's code. (Ask the builder where they're stored. "In the code" is the wrong answer.)
Customers kept apart. Logged in as Customer A, you can't see Customer B's data, even by changing a number in the web address.
Admin pages locked. A regular user who pastes in the address of an admin page gets turned away.
The right people have access. Only current team members can log in, and former employees have been removed.
Only the data it needs. Customer, financial or employee details are stored only if the app truly needs them.

2. Does it actually work?

Start to finish. A brand-new account can complete the main task from start to finish.
It saves. Refresh the page, and everything you did is still there.
It admits failure. Cut the internet mid-task, and the app tells you something went wrong instead of pretending it worked.
One click, one result. Hitting submit twice creates one invoice, order or record, not two.
It catches typos. An obviously wrong entry (40 hours instead of 4) gets flagged before it goes anywhere.
Two people at once. Two people using the app at the same time don't overwrite each other's work.

3. Can anyone besides the builder run it?

Written instructions. Someone else can set up and launch the app from written steps.
Safe changes. We know who is allowed to change the live app, and changes are tried on a test copy before real customers see them.
Alerts. When something breaks, a named person finds out right away.
Working backups. Backups exist, and someone has actually restored one.
A paper trail. The app records who did what and when, so a mistake can be traced back.

4. If it acts on its own (sends, orders or pays)

A person signs off on the risky ones. Large amounts or unusual activity wait for someone to approve them.
Limits. There's a cap on how much the app can do at once, in number of actions and in dollars.
An off switch. Someone non-technical can shut it off immediately.

An app that passes these set of questions can get “assigned” to two owners: a business owner who decides what it should do, and a technical owner who can fix it at 2am (oftentimes this may be the same person).

An app that fails any item stays where it is. It doesn't get new users, new kinds of data or a move up a level until the gap is closed by engineers. Until then, cover the gap with people (for example, someone approves what the app sends before it goes out) and invest in the engineering that makes it scalable and error-proof. Make sure to run the checklist again, before it takes on anything new or gets rolled out broadly.

Looking Ahead

Our fear with vibe-coded applications is that they can lead to an overcorrection. After a bad incident, the portco panics, restricts what its people are allowed to build, and loses the leverage AI was giving it.

We'd much rather see the opposite.

The controller who understands billing better than any software vendor can now build the first version in a weekend, and that's a real edge for a portco. Turning that weekend vibe-code build into something the business can safely run on is the part worth getting engineering help with.

If you have an app your team now depends on, hit reply with what it does and who uses it. On a short email or call we can share our view on how exposed it is, whether it needs a deeper review, and the first few things we'd fix.

Hit reply; we read everything.