Rescue and hardening for AI-built apps

An AI coding assistant optimises for the request in front of it. Asked for a dashboard that shows a user their orders, it will produce exactly that: a query filtered by the current user, rendered in a table. The result works. What it does not do — because nobody asked — is decide what happens when a different user changes the identifier in the request, or when the same webhook arrives twice, or when the database has fifty thousand rows instead of fifty.

These are not bugs in the ordinary sense. Nothing is broken on the path the tool was asked about. The failures live on the paths nobody described: the adversarial one, the concurrent one, the one where a third-party service returns a 500. That is why the same four or five problems appear in app after app regardless of who built them or what they do.

It is also why the work is predictable enough to sell at a fixed price. We are not exploring an unknown codebase hoping to find something. We are checking a known list against a shape we have seen before, and the list is published on this site.

Almost every serious finding reduces to one question: where is the decision made about whether you are allowed to do this? In an app that came out of a builder tool, the answer is usually "in the browser" — the interface hides what you cannot do, and the endpoint behind it assumes the interface did its job.

That is not a security model. It is a user-experience convenience that has been mistaken for one. The fix is unglamorous and mostly mechanical: every privileged action checks the caller on the server; every tenant identifier is resolved from the authenticated session rather than from what the client sent; every table has a policy, and every policy has a test that asserts a user of one tenant cannot reach another’s rows.

We insist on the tests because access control decays silently. A migration adds a table, nobody adds a policy, and the gap does not announce itself — the application keeps working perfectly for every honest user. A test that fails the build is the only thing that catches it.

The common pattern is: a customer pays, the payment provider sends an event, a handler writes a row. Three things go wrong with it. The handler does not verify the event came from the provider, so anyone who finds the URL can grant themselves a subscription. It is not idempotent, so the provider’s ordinary retries create duplicate records. And it handles one event — the successful checkout — so cancellations, refunds and failed renewals never reach your database at all.

The deeper problem is treating your own database as the source of truth about what somebody has paid for. It is not. The payment provider is. What you hold is a cache, and like any cache it needs a way to be rebuilt. We add signature verification and idempotency because they stop new damage, and a reconciliation because it repairs what has already happened.

This matters more than it sounds. By the time a founder notices, there is usually a set of customers whose access does not match their billing in both directions — people paying for something they cannot use, and people using something they stopped paying for.

Error tracking, uptime checks, environment separation, migrations, backups and a tested rollback are the parts most often missing, and they are the parts that decide whether a bad day lasts twenty minutes or a weekend. None of them are visible to a user, which is exactly why a tool asked to build a product never produces them.

The test we apply to a backup is whether it has been restored. An export schedule nobody has ever recovered from is an assumption, and assumptions fail at the worst moment. We execute a restore into a scratch environment during the sprint, so the answer to "can we recover" is a thing that happened rather than a setting that was enabled.