← All field notes
AI products 4 min read

Vetted Software: What It Means Now That AI Can Generate Code

AI has made code faster to produce, but dependable software still requires product judgement, verification and operational ownership.

AI can generate a surprising amount of code in an afternoon. That changes the economics of software development, but it does not remove the difficult part of building a product.

The difficult part is deciding what should exist, choosing the right boundaries, handling the failures nobody demonstrates, and remaining accountable after the first customer arrives. That is what I mean by vetted software.

Vetted software is not code that has merely been reviewed. It is software that has passed through senior product, architecture, security and operational judgement before it is placed in front of customers.

The bottleneck has moved

For years, producing code was one of the slowest parts of product delivery. AI-assisted tools have reduced that constraint. A capable engineer can now explore approaches, create interfaces and connect services much faster than before.

That speed is valuable, but it also makes it easier to create a larger quantity of the wrong thing. DORA’s research describes AI as an amplifier: it magnifies the strengths of effective organisations and the weaknesses of struggling ones. Faster generation does not compensate for unclear outcomes, weak feedback loops or poor operational discipline.

Read the 2025 DORA research on AI-assisted software development.

The new bottleneck is not typing. It is judgement.

Code is an ingredient, not the finished product

A repository can look convincing while the product remains commercially unusable. The interface works on the happy path. The demonstration data is clean. The model responds correctly to the five prompts everyone has tested.

Then production introduces the real questions:

  • What happens when the model is unavailable, slow or unexpectedly expensive?
  • Which actions can a user reverse, and which require approval?
  • How are customer data and secrets separated?
  • Can we understand why a workflow failed without reading raw logs for an hour?
  • Can another team operate the system without depending on its original builder?
  • Does the product create a measurable outcome for the person paying for it?

Those are product questions, engineering questions and business questions at the same time. They cannot be delegated to code generation.

The six tests of vetted software

1. Product fit

The workflow solves a real problem for a defined user. The acceptance criteria describe an outcome rather than a collection of screens. If the team cannot explain who benefits and what changes for them, the specification is not ready.

2. Architectural fit

The system is no more complicated than the problem requires. Important boundaries are explicit, integrations have failure modes, and technology choices reflect the likely scale, skills and operating budget.

3. Security fit

Authentication, authorisation, data handling and dependency risk are designed in rather than added just before launch. For AI systems, this also includes prompt injection, excessive agency, sensitive information disclosure and unsafe tool use. The NIST Secure Software Development Framework provides a useful common language for this work.

4. Reliability fit

The important paths are tested. Failures are contained. Retries, timeouts and fallbacks are deliberate. A deployment can be rolled back. The system is judged on what happens when a dependency misbehaves, not only when everything is healthy.

5. Operational fit

The product has useful logs, metrics, alerts and backups. Someone knows what requires a response and what can wait. Cost can be observed before it becomes a surprise. Routine changes do not require heroics.

6. Ownership fit

The customer owns the code and data. Documentation explains the important decisions. Hosting can be managed, but it is not a trap. If responsibility transfers, the receiving team has what it needs to operate the product.

Where human accountability belongs

AI belongs throughout the delivery process. I use it to analyse briefs, explore options, create tests, challenge assumptions and accelerate implementation. But the final judgement must remain with somebody who can explain the trade-offs and stand behind the result.

At AlanOps, every brief is reviewed personally. I resolve the risky product and technical decisions before proposing delivery. When a build needs additional capacity, I assemble trusted specialists around that defined plan. The software may be AI-assisted; responsibility is never automated away.

What a buyer should expect

A serious delivery partner should be able to show you:

  • which assumptions remain unproven;
  • what is explicitly inside and outside the scope;
  • how quality and launch readiness will be assessed;
  • who owns the code, data and infrastructure;
  • how the system will be supported or handed over; and
  • who is accountable when a difficult decision appears.

If those answers are missing, a fast quote is not certainty. It is merely unpriced risk.


Have a PRD, prototype or product idea?

Send AlanOps the current truth, including what you know, what is still uncertain and what a successful outcome would look like. I will review it personally and recommend the most credible next step.

Submit your product brief.

AS

Written by Alan Son

CTO judgment, grounded in delivery.

I build commercial AI products and help founders turn ambitious technical ideas into reliable production systems.

More about my work

Need an experienced technical partner?

Let’s work out the right next move.